Review of Quantitative Empirical Evaluations of Technology for People with Visual Impairments
Open accessAbstract
Addressing the needs of visually impaired people is of continued interest in Human Computer Interaction (HCI) research. Yet, one of the major challenges facing researchers in this field continues to be how to design adequate quantitative empirical evaluation for these users in HCI. In this paper, we analyse a corpus of 178 papers on technologies designed for people with visual impairments, published since 1988, and including at least one quantitative empirical evaluation (243 evaluations in total). To inform future research in this area, we provide an overview, historic trends and a unified terminology to design and report quantitative empirical evaluations. We identify open issues and propose a set of guidelines to address them. Our analysis aims to facilitate and stimulate future research on this topic.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There exists the bias of comparing a new system to an old one when participants know which system was designed by the researchers.
“Second, there exists the bias of comparing a new system to an old one when participants know which system was designed by the researchers [5, 33].”
From Revamp: Enhancing Accessible Information Seeking Experience of Online Shopping for Blind or Low Vision Users (2021) · cites this paper as [5]
Cites
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433 · see citing works
Measuring Presence in Virtual Environments: A Presence Questionnaire
Witmer, B. G., & Singer, M. J. (1998) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474698565686 · Cited by 6,371 · see citing works
Designing the user interface: Strategies for effective human-computer interaction
(1993) · Applied Ergonomics · doi:10.1016/0003-6870(93)90471-k · Cited by 5,592 · see citing works
(2004) · Cited by 5,198 · see citing works
An Empirical Evaluation of the System Usability Scale
Bangor, A., Kortum, P. T., & Miller, J. T. (2008) · International Journal of Human–Computer Interaction · doi:10.1080/10447310802205776 · Cited by 5,181 · see citing works
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297 · see citing works
Global estimates of visual impairment: 2010
Pascolini, D., & Mariotti, S. P. (2011) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2011-300539 · Cited by 3,773 · see citing works
Heuristic evaluation of user interfaces
Nielsen, J., & Molich, R. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97281 · Cited by 3,533 · see citing works
Nasa-task load index (NASA-TLX); 20 years later: (577632012-009)
Hart, S. G. (2006) · PsycEXTRA Dataset · doi:10.1037/e577632012-009 · Cited by 838 · see citing works
Usability evaluation considered harmful (some of the time)
Greenberg, S., & Buxton, B. (2008) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1357054.1357074 · Cited by 601 · see citing works
The Robotic Social Attributes Scale (RoSAS): Development and Validation
Carpinella, C. M., Wyman, A. B., Perez, M. A., & Stroessner, S. J. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020208 · Cited by 569 · see citing works
Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios
Schrepp, M., Hinderks, A., & Thomaschewski, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-07668-3_37 · Cited by 560 · see citing works
Boehner, K., Vertesi, J., Sengers, P., & Dourish, P. (2007) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240789 · Cited by 533 · see citing works
Comparison of three one-question, post-task usability questionnaires
Sauro, J., & Dumas, J. S. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518946 · Cited by 511 · see citing works
Balancing the strengths of systematic and narrative reviews
Collins, J. A., & Fauser, B. C. J. M. (2004) · Human Reproduction Update · doi:10.1093/humupd/dmh058 · Cited by 511 · see citing works
Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation Methods
Gray, W. D., & Salzman, M. (1998) · Human-Computer Interaction · doi:10.1207/s15327051hci1303_2 · Cited by 486 · see citing works
Editorial Perspective: The use of person‐first language in scholarly writing may accentuate stigma
Gernsbacher, M. A. (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12706 · Cited by 428 · see citing works
Assistive Technology for Visually Impaired and Blind People
Hersh, M. A., & Johnson, M. A. (2008) · Springer eBooks · doi:10.1007/978-1-84628-867-8 · Cited by 426 · see citing works
Research contributions in human-computer interaction
Wobbrock, J. O., & Kientz, J. A. (2016) · interactions · doi:10.1145/2907069 · Cited by 420 · see citing works
The Promise of Empathy: Design, Disability, and Knowing the "Other"
Bennett, C. L., & Rosner, D. K. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300528 · Cited by 413 · see citing works
The diversity of participatory design research practice at PDC 2002–2012
Halskov, K., & Hansen, N. B. (2015) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2014.09.003 · Cited by 365 · see citing works
Fair Statistical Communication in HCI
Dragicevic, P. (2016) · Human-computer interaction series · doi:10.1007/978-3-319-26633-6_13 · Cited by 339 · see citing works
METHODOLOGY MATTERS: DOING RESEARCH IN THE BEHAVIORAL and SOCIAL SCIENCES
McGrath, J. E. (1995) · Elsevier eBooks · doi:10.1016/b978-0-08-051574-8.50019-4 · Cited by 296 · see citing works
Bhowmick, A., & Hazarika, S. M. (2017) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-016-0235-6 · Cited by 276 · see citing works
Maidenbaum, S., Abboud, S., & Amedi, A. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.11.007 · Cited by 230 · see citing works
Spatial navigation by congenitally blind individuals
Schinazi, V. R., Thrash, T., & Chebat, D. (2016) · WIREs Cognitive Science · doi:10.1002/wcs.1375 · Cited by 227 · see citing works
The Evaluation of Accessibility, Usability, and User Experience
Petrie, H., & Bevan, N. (2009) · Human factors and ergonomics · doi:10.1201/9781420064995-c20 · Cited by 185 · see citing works
From Mice to Men - 24 Years of Evaluation in CHI
Barkhuus, L., & Rode, J. A. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.2180963 · Cited by 180 · see citing works
The Evaluation of Accessibility, Usability, and User Experience
Stephanidis, C. (2009) · CRC Press eBooks · doi:10.1201/9781420064995-31 · Cited by 171 · see citing works
Literature Reviews and the Hermeneutic Circle
Boell, S. K., & Ćećez-Kecmanović, D. (2010) · Australian Academic & Research Libraries · doi:10.1080/00048623.2010.10721450 · Cited by 167 · see citing works
Sato, D., Oh, U., Naito, K., Takagi, H., Kitani, K., & Asakawa, C. (2017) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3132525.3132535 · Cited by 159 · see citing works
Kjeldskov, J., & Skov, M. B. (2014) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/2628363.2628398 · Cited by 143 · see citing works
Gori, M., Cappagli, G., Tonelli, A., Baud‐Bovy, G., & Finocchietti, S. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.043 · Cited by 133 · see citing works
CASE STUDIES, SINGLE-SUBJECT RESEARCH, AND N OF 1 RANDOMIZED TRIALS
Backman, C. L., & Harris, S. R. (1999) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/00002060-199903000-00022 · Cited by 118 · see citing works
Kayukawa, S., Higuchi, K., Guerreiro, J., Morishima, S., Sato, Y., Kitani, K., & Asakawa, C. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300282 · Cited by 115 · see citing works
Albouys-Perrois, J., Laviole, J., Briant, C., & Brock, A. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174203 · Cited by 107 · see citing works
Vashistha, A., Cutrell, E. B., Borriello, G., & Thies, W. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702191 · Cited by 98 · see citing works
Virtual Navigation for Blind People
Guerreiro, J., Ahmetovic, D., Kitani, K., & Asakawa, C. (2017) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3132525.3132545 · Cited by 98 · see citing works
Micro-ethics for participatory design with marginalised children
Spiel, K., Brulé, É., Frauenberger, C., Bailly, G., & Fitzpatrick, G. (2018) · Participatory Design Conference · doi:10.1145/3210586.3210603 · Cited by 97 · see citing works
Understanding Low Vision People's Visual Perception on Commercial Augmented Reality Glasses
Zhao, Y., Hu, M., Hashash, S., & Azenkot, S. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025949 · Cited by 95 · see citing works
Representing users in accessibility research
Sears, A., & Hanson, V. L. (2012) · ACM Transactions on Accessible Computing · doi:10.1145/2141943.2141945 · Cited by 79 · see citing works
Ducasse, J., Macé, M. J. ‐., Serrano, M., & Jouffrais, C. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858058 · Cited by 78 · see citing works
Self-citationHong, J., Pradhan, A., Froehlich, J. E., & Findlater, L. (2017) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3132525.3132538 · Cited by 77 · see citing works
Kelly, S. M., & Smith, D. W. (2011) · Journal of Visual Impairment & Blindness · doi:10.1177/0145482x1110500205 · Cited by 75 · see citing works
Vacher, M., Caffiau, S., Portet, F., Meillon, B., Roux, C., Elias, E., Lecouteux, B., & Chahuara, P. (2015) · ACM Transactions on Accessible Computing · doi:10.1145/2738047 · Cited by 73 · see citing works
Agency in Assistive Technology Adoption
Pal, J., Viswanathan, A., Chandra, P., Nazareth, A., Kameswaran, V., Subramonyam, H., Johri, A., Ackerman, M. S., & O’Modhrain, S. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025895 · Cited by 69 · see citing works
DEEP: Design-Oriented Evaluation of Perceived Usability
Yang, T., Linder, J., & Bolchini, D. P. (2011) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2011.586320 · Cited by 63 · see citing works
Perer, A., & Shneiderman, B. (2009) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2009.44 · Cited by 60 · see citing works
Alswailmi, F. K. (2018) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.343.14510 · Cited by 57 · see citing works
Swaminathan, S., Roumen, T. J., Kovács, R., Stangl, D., Mueller, S., & Baudisch, P. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858245 · Cited by 53 · see citing works
Seven Guiding Scenarios for Information Visualization Evaluation
Lam, H., Bertini, E., Isenberg, P., Catherine Plaisant, B., & Carpendale, S. (2011) · doi:10.11575/prism/30512 · Cited by 52 · see citing works
Spiel, K., Frauenberger, C., Hornecker, E., & Fitzpatrick, G. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025785 · Cited by 52 · see citing works
Wang, R., Yu, C., Yang, X.-D., He, W., & Shi, Y. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300254 · Cited by 48 · see citing works
Evaluating touch-screen vibration modality for blind users to access simple shapes and graphics
Tekli, J., Issa, Y. B., & Chbeir, R. (2017) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2017.10.009 · Cited by 46 · see citing works
Environmental Factors in Indoor Navigation Based on Real-World Trajectories of Blind Users
Kacorri, H., Ohn-Bar, E., Kitani, K., & Asakawa, C. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173630 · Cited by 46 · see citing works
A Large Inclusive Study of Human Listening Rates
Bragg, D., Bennett, C., Reinecke, K., & Ladner, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174018 · Cited by 43 · see citing works
An approach to the evaluation of assistive technology
Stevens, R., & Edwards, A. D. N. (1996) · doi:10.1145/228347.228359 · Cited by 43 · see citing works
Inclusive Education Technologies
Metatla, O., Serrano, M., Jouffrais, C., Thieme, A., Kane, S. K., Branham, S. M., Brulé, É., & Bennett, C. L. (2018) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3170427.3170633 · Cited by 42 · see citing works
Self-citationEvaluation of a non-visual molecule browser
Brown, A. J., Pettifer, S. R., & Stevens, R. (2003) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1028630.1028639 · Cited by 39 · see citing works
Usage of Subjective Scales in Accessibility Research
Trewin, S., Marques, D., & Guerreiro, T. (2015) · Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS '15 · doi:10.1145/2700648.2809867 · Cited by 35 · see citing works
Parker, A. T. (2009) · Journal of Visual Impairment & Blindness · doi:10.1177/0145482x0910300607 · Cited by 34 · see citing works
Indoor navigation for people who are blind or vision impaired: where are we and where are we going?
Gallagher, T. J., Wise, E., Yam, H., Li, B.-H., Ramsey-Stewart, E., Dempster, A., & Rizos, C. (2014) · Journal of Location Based Services · doi:10.1080/17489725.2014.895062 · Cited by 34 · see citing works
Pictures in Your Mind: Using Interactive Gesture-Controlled Reliefs to Explore Art
Reichinger, A., Carrizosa, H. G., Wood, J., Schröder, S., Löw, C., Luidolt, L. R., Schimkowitsch, M., Fuhrmann, A., Maierhofer, S., & Purgathofer, W. (2018) · ACM Transactions on Accessible Computing · doi:10.1145/3155286 · Cited by 34 · see citing works
Doush, I. A., Pontelli, E., Simon, D. A., Son, T. C., & Ma, O. (2009) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1639642.1639669 · Cited by 33 · see citing works
Bullseye! when Fitts' law doesn't fit
Friedlander, N., Schlueter, K., & Mantei, M. M. (1998) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/274644.274681 · Cited by 33 · see citing works
From screen reading to aural glancing: towards instant access to key page sections
Gadde, P., & Bolchini, D. (2014) · Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility - ASSETS '14 · doi:10.1145/2661334.2661363 · Cited by 30 · see citing works
Grow, S. L. (1999) · PubMed · Cited by 28 · see citing works
Bardot, S., Serrano, M., & Jouffrais, C. (2016) · Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/2935334.2935342 · Cited by 28 · see citing works
Self-citationIncluding blind people in computing through access to graphs
Balik, S. P., Mealin, S. P., Stallmann, M. F. M., Rodman, R. D., Glatz, M., & Sigler, V. J. (2014) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2661334.2661364 · Cited by 27 · see citing works
Parente, P. (2003) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1028630.1028633 · Cited by 25 · see citing works
Soviak, A., Borodin, A., Ashok, V. G., Borodin, Y., Puzis, Y., & Ramakrishnan, I. V. (2016) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2982142.2982175 · Cited by 23 · see citing works
Exploring Auditory Graphing Software in the Classroom
Tomlinson, B. J., Batterman, J. M., Chew, Y., Henry, A. G., & Walker, B. (2016) · ACM Transactions on Accessible Computing · doi:10.1145/2994606 · Cited by 23 · see citing works
Self-citationToward classroom experiences inclusive of students with disabilities
Metatla, O., Thieme, A., Brulé, É., Bennett, C. L., Serrano, M., & Jouffrais, C. (2018) · interactions · doi:10.1145/3289485 · Cited by 22 · see citing works
Self-citationAccessible Gesture Typing for Non-Visual Text Entry on Smartphones
Billah, S. M., Ko, Y.-J., Ashok, V., Bi, X., & Ramakrishnan, IV (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300606 · Cited by 21 · see citing works
Langdon, P., Lazar, J. K., Heylighen, A., & Dong, H. (2016) · Springer eBooks · doi:10.1007/978-3-319-29498-8 · Cited by 21 · see citing works
Ducasse, J., Macé, M. J. ‐., Oriola, B., & Jouffrais, C. (2018) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3204460 · Cited by 21 · see citing works
Self-citationAutonomous Selection and Printing of 3D Models for People Who Are Blind
Götzelmann, T. (2018) · ACM Transactions on Accessible Computing · doi:10.1145/3241066 · Cited by 17 · see citing works
On the Evaluation of Novel Sonification Techniques for Non-Visual Shape Exploration
Mascetti, S., Gerino, A., Bernareggi, C., & Picinali, L. (2017) · ACM Transactions on Accessible Computing · doi:10.1145/3046789 · Cited by 17 · see citing works
Walker, B. N., Lindsay, J., & Godfrey, J. (2003) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1028630.1028634 · Cited by 17 · see citing works
Using Dynamic Audio Feedback to Support Peripersonal Reaching in Young Visually Impaired People
Wilson, G., & Brewster, S. A. (2016) · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2982142.2982160 · Cited by 16 · see citing works
Evaluation of a non-visual molecule browser
Brown, A. J., Pettifer, S. R., & Stevens, R. (2003) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/1029014.1028639 · Cited by 12 · see citing works
Investigating the Mode in Multimodal Video Games
Darin, T., Andrade, R. M. C., Merabet, L. B., & Sánchez, J. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3053177 · Cited by 11 · see citing works
Using Dynamic Audio Feedback to Support Peripersonal Reaching in Visually Impaired People
Wilson, G., & Brewster, S. A. (2015) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2700648.2811339 · Cited by 10 · see citing works
Manzoor, A., Arooj, S., Zulfiqar, S., Parvez, M., Shahid, S., & Karim, A. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300734 · Cited by 10 · see citing works
Investigating Feedback for Two-Handed Exploration of Digital Maps Without Vision
Bardot, S., Serrano, M., Perrault, S. T., Zhao, S., & Jouffrais, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29381-9_19 · Cited by 9 · see citing works
Self-citationEvaluating Technologies with and for Disabled Children
Brulé, É., Metatla, O., Spiel, K., Kharrufa, A., & Robinson, C. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3311757 · Cited by 7 · see citing works
Self-citationIntegrating statistics and visualization for exploratory power
PererAdam, & ShneidermanBen (2009) · IEEE Computer Graphics and Applications · doi:10.5555/1669295.1669302 · Cited by 5 · see citing works
Walker, B. N., Lindsay, J., & Godfrey, J. (2003) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/1029014.1028634 · Cited by 4 · see citing works
Flagged first, then by citations
1 of the 89 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Dudley, J. J., Yin, L., Garaj, V., & Kristensson, P. O. (2023) · Virtual Reality · doi:10.1007/s10055-023-00850-8 · Cited by 225 · see citing works
Sharif, A., McCall, A. L., & Bolante, K. R. (2022) · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3517428.3544813 · Cited by 85 · see citing works
Mukhiddinov, M., & Kim, S.-Y. (2021) · Processes · doi:10.3390/pr9101726 · Cited by 82 · see citing works
Butler, M., Holloway, L. M., Reinders, S., Goncu, C., & Marriott, K. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445207 · Cited by 73 · see citing works
Literature Reviews in HCI: A Review of Reviews
Stefanidi, E., Bentvelzen, M., Woźniak, P. W., Kosch, T., Woźniak, M. P., Mildner, T., Schneegass, S., Müller, H., & Niess, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581332 · Cited by 66 · see citing works
Mapping HCI research methods for studying social media interaction: A systematic literature review
Shibuya, Y., Hamm, A., & Pargman, T. C. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.107131 · Cited by 55 · see citing works
A systematic review of extended reality (XR) for understanding and augmenting vision loss
Kasowski, J., Johnson, B. A., Neydavood, R., Akkaraju, A., & Beyeler, M. (2023) · Journal of Vision · doi:10.1167/jov.23.5.5 · Cited by 47 · see citing works
State of the Art in AAC: A Systematic Review and Taxonomy
Curtis, H., Neate, T., & Gonzalez, C. V. (2022) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3517428.3544810 · Cited by 47 · see citing works
Gamage, B., Do, T.-T., Price, N. S. C., Lowery, A., & Marriott, K. (2023) · The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3608955 · Cited by 39 · see citing works
Nanavati, A., Ranganeni, V., & Çakmak, M. (2023) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-062823-024352 · Cited by 44 · see citing works
Wang, R., Chen, Z., Zhang, M. R., Li, Z., Liu, Z., Dang, Z., Yu, C., & Chen, X. '. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445547 · Cited by 42 · see citing works
Hwang, H., Jung, H.-T., Giudice, N. A., Biswas, J., Lee, S. I., & Kim, D. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642181 · Cited by 35 · see citing works
Mapping 20 years of accessibility research in HCI: A co-word analysis
Sarsenbayeva, Z., van Berkel, N., Hettiachchi, D., Tag, B., Velloso, E., Gonçalves, J., & Kostakos, V. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103018 · Cited by 33 · see citing works
Taking Stock of Concept Inventories in Computing Education: A Systematic Literature Review
Ali, M., Ghosh, S., Rao, P. V., Dhegaskar, R., Jawort, S., Medler, A., Shi, M., & Dasgupta, S. (2023) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3568813.3600120 · Cited by 26 · see citing works
Nagassa, R. G., Butler, M., Holloway, L. M., Goncu, C., & Marriott, K. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581389 · Cited by 24 · see citing works
A Large-Scale Mixed-Methods Analysis of Blind and Low-vision Research in ACM and IEEE
Thoo, Y.-J., Medina, M. J., Froehlich, J. E., Ruffieux, N., & Lalanne, D. (2023) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3597638.3608412 · Cited by 27 · see citing works
Rogers, K., Hirzle, T., Karaosmanoglu, S., Palomino, P. T., Durmanova, E., Isotani, S., & Nacke, L. E. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3685266 · Cited by 24 · see citing works
Zhao, K., Bardot, S., Serrano, M., Simonnet, M., Oriola, B., & Jouffrais, C. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445578 · Cited by 21 · see citing works
Self-citationOrtloff, A.-M., Martius, F., Meier, M., Sans-Raimbault, T., Geierhaas, L., & Smith, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713671 · Cited by 19 · see citing works
VibHand: On-Hand Vibrotactile Interface Enhancing Non-Visual Exploration of Digital Graphics
Zhao, K., Serrano, M., Oriola, B., & Jouffrais, C. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3427335 · Cited by 18 · see citing works
Self-citationNevsky, A., Neate, T., Simperl, E. P. B., & Vatavu, R.-D. (2023) · ACM International Conference on Interactive Media Experiences · doi:10.1145/3573381.3596159 · Cited by 18 · see citing works
Jeong, I., Kim, K., Jung, J., & Cho, J. (2025) · Electronics · doi:10.3390/electronics14030425 · Cited by 15 · see citing works
Alonzo, O., & Hassan, S. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746355 · Cited by 13 · see citing works
Deep Learning-Based Optimization of Visual–Auditory Sensory Substitution
Kim, M., Park, Y., Moon, K., & Jeong, C. Y. (2023) · IEEE Access · doi:10.1109/access.2023.3243641 · Cited by 12 · see citing works
“Many previous studies have investigated the perceptual experience of sensory substitution based on behavioral observations of the participants [19], [20], [21], [24], [25].”
BackgroundCLARKE, J. M., Mehrnezhad, M., & Toreini, E. (2024) · ACM Transactions on Accessible Computing · doi:10.1145/3641281 · Cited by 12 · see citing works
McDonnell, E. J., & Findlater, L. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675649 · Cited by 13 · see citing works
Morrison, C. P., Cutrell, E. B., Grayson, M., Becker, E. R., Kladouchou, V., Pring, L., Jones, K. M., Marques, R. F., Longden, C., & Sellen, A. J. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3471210 · Cited by 11 · see citing works
A Participatory Design Approach to Creating Echolocation-Enabled Virtual Environments
Andrade, R., Baker, S., Waycott, J., & Vetere, F. (2022) · ACM Transactions on Accessible Computing · doi:10.1145/3516448 · Cited by 10 · see citing works
Fayyaz, N., Khusro, S., & Uddin, I. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.101617 · Cited by 10 · see citing works
Human Robot Interaction for Blind and Low Vision People: A Systematic Literature Review
Wei, Y., Rocher, N., Gupta, C., Nguyen, M. H., Zimmermann, R., Ooi, W. T., Jouffrais, C., & Nanayakkara, S. C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713438 · Cited by 8 · see citing works
Self-citationAn Educational Dialogue System for Visually Impaired People
Balestrucci, P. F., Nuovo, E. D., Sanguinetti, M., Luca Anselma, A., Bernareggi, C., & Mazzei, A. (2024) · IEEE Access · doi:10.1109/access.2024.3479883 · Cited by 8 · see citing works
Mack, K., McDonnell, E., Jain, D., Lu Wang, L., E. Froehlich, J., & Findlater, L. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445412 · Cited by 7 · see citing works
Abe, Y., Matsushima, K., Hara, K., Sakamoto, D., & Ono, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714284 · Cited by 7 · see citing works
Faisandaz, G. R. J., Goguey, A., Jouffrais, C., & Nigay, L. (2022) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3536221.3556589 · Cited by 7 · see citing works
Self-citationKhasawneh, M. A. S., & Khasawneh, M. A. S. (2023) · Journal of Southwest Jiaotong University · doi:10.35741/issn.0258-2724.58.4.77 · Cited by 7 · see citing works
Kim, M., Park, Y., Moon, K.-D., & Jeong, C. Y. (2024) · IEEE Access · doi:10.1109/access.2024.3419102 · Cited by 7 · see citing works
Tactile maps of historical gardens: method of content selection
Zwirowicz-Rutkowska, A., Mościcka, A., Araszkiewicz, A., Wabiński, J., & Kiliszek, D. (2023) · Heritage Science · doi:10.1186/s40494-023-00886-8 · Cited by 7 · see citing works
Challenges in Mobile Text Entry using Virtual Keyboards for Low-Vision Users
Stefanis, V., Komninos, A., & Garofalakis, J. (2020) · 19th International Conference on Mobile and Ubiquitous Multimedia · doi:10.1145/3428361.3428391 · Cited by 6 · see citing works
Widen your vision: from technical accessibility to semantic intelligibility of information
Baudet, C., Medina, M. J., & Benoit, C. (2023) · Universal Access in the Information Society · doi:10.1007/s10209-022-00963-y · Cited by 7 · see citing works
Studies on blind and visually impaired users in LIS literature: A review of research methods
Xie, I., Wang, S., & Saba, M. (2021) · Library & Information Science Research · doi:10.1016/j.lisr.2021.101109 · Cited by 6 · see citing works
Sharif, A., Pruekcharoen, P., Ramesh, T., Shang, R., Williams, S., & Hsieh, G. (2022) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3517428.3550359 · Cited by 6 · see citing works
Zhao, K., Mulet, J., Sorita, C., Oriola, B., Serrano, M., & Jouffrais, C. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3567733 · Cited by 6 · see citing works
Self-citationFinding Gaps in Modern Visualization LiteracyPreprint
Creamer, M. M., Padilla, L., & Borkin, M. A. (2024) · doi:10.31219/osf.io/jy9v2 · Cited by 6 · see citing works
Methodical testing of tactile cartographic signs in isolation and in context
Mościcka, A., Śmiechowska-Petrovskij, E., Wabiński, J., Araszkiewicz, A., & Kiliszek, D. (2024) · Cartography and Geographic Information Science · doi:10.1080/15230406.2024.2379456 · Cited by 5 · see citing works
Evaluation of emerging educational technologies in the classroom
Kharrufa, A., Abacı, S., Brulé, É., Slovák, P., Venn-Wycherley, M., & Lambton-Howard, D. (2020) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3397617.3398065 · Cited by 4 · see citing works
Langer, D., Legler, F., Diekmann, P., Dettmann, A., Glende, S., & Bullinger, A. C. (2024) · Robotics · doi:10.3390/robotics13030043 · Cited by 3 · see citing works
Haz, L., Marquino, J. M. M., & Ramos, I. D. R. B. (2024) · Ciencia Latina Revista Científica Multidisciplinar · doi:10.37811/cl_rcm.v8i4.12545 · Cited by 3 · see citing works
Medina, M. J., Baudet, C., & Lalanne, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05028-2_1 · Cited by 3 · see citing works
Medina, M. J., Thoo, Y.-J., Baudet, C., Froehlich, J. E., Ruffieux, N., & Lalanne, D. (2025) · ACM Transactions on Accessible Computing · doi:10.1145/3726531 · Cited by 3 · see citing works
Information Security Fatigue in Visually Impaired University Students
Kakinouchi, M., & Omote, K. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-030-99619-2_43 · Cited by 3 · see citing works
Electromagnetic actuation for a vibrotactile display: Assessing stimuli complexity and usabilityPreprint
Proulx, M. J., Eracleous, T., Spencer, B., Passfield, A., de Sousa, A. A., & Mohammadi, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.13295 · Cited by 3 · see citing works
Vargas, P. N., & Trevisan, D. G. (2024) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3702038.3702105 · Cited by 2 · see citing works
Furthering Visual Accessibility with Extended Reality (XR): A Systematic Review.Preprint
Kasowski, J., Johnson, B. A., Neydavood, R., Akkaraju, A., & Beyeler, M. (2021) · arXiv (Cornell University) · Cited by 2 · see citing works
Zambrano, A., Salvador, M. N., Grijalva, F., Mora, H. C., & Garzón, N. O. (2024) · Technologies · doi:10.3390/technologies12080133 · Cited by 1 · see citing works
Zhu, S., Nguyen, M. T. D., Mascelloni, M., Zhu, H. Y., Chia, R., Singh, A. K., Lin, C., & Jin, C. T. (2025) · Immersive and 3D Audio: from Architecture to Automotive (I3DA) · doi:10.1109/i3da65421.2025.11202115 · Cited by 1 · see citing works
The Accessibility, Security, and Privacy Nexus: Trends and Opportunities
Mack, K., Chen, Y.-J., Zhang, L., Gurari, D., Sharma, T., Wang, Y., & Findlater, L. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3759688 · Cited by 1 · see citing works
Sipatchin, A., García, M. G., Sauer, Y., & Wahl, S. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19159571 · Cited by 1 · see citing works
AGAA: An Android GUI Accessibility Adapter for Low Vision Users
Xu, Y., Li, Z., Liu, H., & Liu, Y. (2023) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac57700.2023.00062 · Cited by 1 · see citing works
Wilson, S. L., Sreedharam, R. K., & Ramayah, B. (2023) · Springer eBooks · doi:10.1007/978-981-99-5076-8_4 · Cited by 1 · see citing works
Mathematics Model Interest and Actual Use of The ABC Digital Wallet
Waluo, M., Islami, M. C. P., Winursito, Y. C., Wayulo, M. R., & Marwadelia, A. (2023) · Nusantara Science and Technology Proceedings · doi:10.11594/nstp.2023.3631
A Survey on - Accessible Visual Information Retrieval (AVIR)
Ms. Anu Mohan, M., Santhosh Krishna, B., V, Simi Roshan, S., Reddy, Y. L., & S, J. K. G. (2024) · International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV) · doi:10.1109/icicv62344.2024.00073
Reducing Security Fatigue in Visually Impaired University Students through Problem Posing
Kakinouchi, M., & Omote, K. (2024) · Journal of Information Processing · doi:10.2197/ipsjjip.32.618
Evaluating Effects of Continuous Occlusion on Visual Letter Identification
Fan, Y., Lyu, X., Lyu, S., Li, B. J., Guo, J., Zhao, Y., & Zhou, F. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2402571
Hamash, M., Khan, M. R., & Tiernan, P. (2026) · AI Computer Science and Robotics Technology · doi:10.5772/acrt20250217
Beyond Additive Design: An Empirical Taxonomy of Multimodal STEM Accessibility Systems
Sadallah, M., & Encelle, B. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799343
Puri, P. R., Annan, D., Mejía-Romero, S., Kabore, S. S., Gingras-Royer, N., Nguyen, K. K., & Nemargut, J. P. (2026) · doi:10.2196/preprints.99266
Laufer, J., Banh, L., Schoormann, T., & Strobel, G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-28316-0_7
Stanczyk, A., Tran, M., Saka, T., Zou, Y., & Moonsamy, V. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812887
Qin, Y., Zhang, Y., Desai, N., Karimi, H. A., Huang, K.-T., & Du, N. (2026) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2026.2685237
Tian, J., Gao, T., Wu, D., & Zhou, H. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2695933
Generating clothing graphic captions for visually impaired users
Alluqmani, A., Harvey, M., & Paramita, M. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21774-w
The Work to Make Assistance Work: Learning from Videos by Visually Impaired Content Creators
Wester, J., Rudaz, D., Khan, M. H., Due, B. L., & Brown, B. A. T. (2026) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3829807.3829822
Puri, P. R., Annan, D., Mejía-Romero, S., Kabore, S. S., Gingras-Royer, N., Nguyen, K. K., & Nemargut, J. P. (2026) · JMIR Human Factors · doi:10.2196/99266
- Authors
- Brulé, E., Tomlinson, B. J., Metatla, O., Jouffrais, C., & Serrano, M.
- Affiliations
- University of Sussex, Brighton, United Kingdom
- Georgia Institute of Technology (US)
- University of Bristol (GB)
- Centre National de la Recherche Scientifique (FR)
- Université de Toulouse (FR)
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-14
- OpenAlex
- W3000122771
- Cited by
- 76
- References
- 89
- Citations by year
- 2026: 10 · 2025: 8 · 2024: 15 · 2023: 18 · 2022: 9 · 2021: 13 · 2020: 3
- FWCI
- 14.51
- Type
- conference paper
- Language
- en
- Open access
- green
- hal.science
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Sussex Research Online (University of Sussex) (submittedVersion)
- HAL (submittedVersion, other-oa)
- Figshare (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/N00616X/2)
- University of Sussex (RDF round 10)
- Agence Nationale de la Recherche (ANR-14-CE17-0018)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- visual impairment, assistive technology, evaluation guidelines, evaluation terminology, accessibility, research trends
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
