The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Comparing evaluation techniques for text readability software for adults with intellectual disabilities
Abstract
In this paper, we compare alternative techniques for evaluating a software system for simplifying the readability of texts for adults with mild intellectual disabilities (ID). We introduce our research on the development of software to automatically simplify news articles, display them, and read them aloud for adults with ID. Using a Wizard-of-Oz prototype, we conducted experiments with a group of adults with ID to test alternative formats of questions to measure comprehension of the information in the news articles. We have found that some forms of questions work well at measuring the difficulty level of a text: multiple-choice questions with three answer choices, each illustrated with clip-art or a photo. Some types of questions do a poor job: yes/no questions and Likert-scale questions in which participants report their perception of the text's difficulty level. Our findings inform the design of future evaluation studies of computational linguistic software for adults with ID; this study may also be of interest to researchers conducting usability studies or other surveys with adults with ID.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The reading test is a commonly used test in measuring readability or easiness of reading.
“The evaluation method used in this study was a reading test because this is a commonly used test in measuring readability or easiness of reading (De Lange, Esterhuizen & Beatty, 1993;Huenerfauth, Feng & Elhadad, 2009).”
From Reading on the Computer Screen: Does Font Type has Effects on Web Text Readability? (2013) · cites this paper as Huenerfauth, Feng & Elhadad, 2009
Citations
Cites
(1998) · Cited by 4,875
Cognitive Processes in Comprehension
Just, M. A., & Carpenter, P. A. (2013) · Psychology Press eBooks · doi:10.4324/9781315798851 · Cited by 1,150
Kincaid, J. P., Robert, P. F. L., Richard, L. R., & Brad, S. (1975) · Cited by 739
(1995) · Medical Entomology and Zoology · Cited by 706
Handbook of Mental Retardation and Development
Kugel, R. B. (1999) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/00004703-199906000-00014 · Cited by 320
Discourse Comprehension and Sources of Individual Differences.
Perfetti, C. A., & Lesgold, A. M. (1977) · Cited by 318
On the Failure of Readability Formulas to Define Readable Texts: A Case Study from Adaptations
Davison, A. L., & Kantor, R. N. (1982) · Reading Research Quarterly · doi:10.2307/747483 · Cited by 288
Wong, B. Y. L., & Jones, W. (1982) · Learning Disability Quarterly · doi:10.2307/1510290 · Cited by 256
Metacomprehension of text material.
Maki, R. H., & Berry, S. L. (1984) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.10.4.663 · Cited by 243
Handbook of Mental Retardation and Development
Burack, J. A., Hodapp, R. M., & Zigler, E. (1998) · Cited by 210
A machine learning approach to reading level assessment
Petersen, S. E., & Mari Ostendorf, E. (2008) · Computer Speech & Language · doi:10.1016/j.csl.2008.04.003 · Cited by 203
A statistical model for scientific readability
Si, L., & Callan, J. (2001) · Proceedings of the tenth international conference on Information and knowledge management · doi:10.1145/502585.502695 · Cited by 197
Heilman, M., Collins‐Thompson, K., Callan, J., & Eskénazi, M. (2007) · Cited by 188
Gray, W. S., & Leary, B. E. (1936) · The English Journal · doi:10.2307/805207 · Cited by 179
Simplifying Text for Language-Impaired Readers.
Carroll, J., Minnen, G., Pearce, D., Canning, Y., Devlin, S. L., & Tait, J. I. (1999) · Cited by 154
Cognitively motivated features for readability assessment
Feng, L., Elhadad, N., & Huenerfauth, M. (2009) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/1609067.1609092 · Cited by 140
Self-citationKatims, D. S. (2000) · Education and training in mental retardation and developmental disabilities · doi:10.1177/215416470003500101 · Cited by 133
Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2001) · Education and training in mental retardation and developmental disabilities · doi:10.1177/215416470103600110 · Cited by 103
Language in Mental Retardation
Rondal, J. A., & Edwards, S. (1996) · Cited by 82
Web accessibility for people with cognitive disabilities
Small, J., Schallau, P., Brown, K., & Appleyard, R. (2005) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1056808.1057024 · Cited by 66
Computer usage by young individuals with down syndrome
Feng, J. H., Lazar, J. K., Kumin, L., & Ozok, A. A. (2008) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1414471.1414480 · Cited by 57
Symbols can improve the reading comprehension of adults with learning disabilities
Jones, F. W., Long, K., & Finlay, W. M. L. (2007) · Journal of Intellectual Disability Research · doi:10.1111/j.1365-2788.2006.00926.x · Cited by 46
Understanding mobile phone requirements for young adults with cognitive disabilities
Dawe, M. (2007) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1296843.1296874 · Cited by 44
Usability evaluation involving participants with cognitive disabilities
Lepistö, A., & Ovaska, S. (2004) · Nordic Conference on Human-Computer Interaction · doi:10.1145/1028014.1028061 · Cited by 44
Generating readable texts for readers with low basic skills
Williams, S., & Reiter, E. (2005) · Open Research Online (The Open University) · Cited by 38
Assessing the reading comprehension of adults with learning disabilities
Jones, F. W., Long, K., & Finlay, W. M. L. (2006) · Journal of Intellectual Disability Research · doi:10.1111/j.1365-2788.2006.00787.x · Cited by 35
Evaluation of a psycholinguistically motivated timing model for animations of american sign language
Huenerfauth, M. (2008) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1414471.1414496 · Cited by 35
Self-citationDavies, D. K., Stock, S. E., King, L. R., & Wehmeyer, M. L. (2008) · Intellectual and developmental disabilities · doi:10.1352/1934-9556(2008)46[290:mimfea]2.0.co;2 · Cited by 33
Cohen, S. S., & Steinberg, J. E. (1983) · Reading Research Quarterly · doi:10.2307/747339 · Cited by 18
Comprehension and Mental Retardation
Bilsky, L. H. (1985) · International review of research in mental retardation · doi:10.1016/s0074-7750(08)60237-2 · Cited by 11
LoPresti, E., Kirsch, N. L., Simpson, R. C., & Schreckenghost, D. L. (2005) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1090785.1090824 · Cited by 6
Fickas, S. F., Pataky, C., & Chen, Z. (2006) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1168987.1169056 · Cited by 2
Flagged first, then by citations
Research works printed but not linked: 5
- Drew , C.J. , and Hardman , M.L . 2004 . Mental retardation: A lifespan approach to people with intellectual disabilities ( 8 th ed.). Columbus, OH: Merrill. Drew, C.J., and Hardman, M.L. 2004. Mental retardation: A lifespan approach to people with intellectual disabilities (8th ed.). Columbus, OH: Merrill.
- 10.5555/1609067.1609092 · DOI
- 10.1037/0278-7393.10.4.663 · DOI
- TheArcLink . 2003. The Desk Launches in 11 States for People with Disabilities, (accessed October 29, 2008 ), http://www.thearclink.org/news/article.asp?ID=591 TheArcLink. 2003. The Desk Launches in 11 States for People with Disabilities, (accessed October 29, 2008), http://www.thearclink.org/news/article.asp?ID=591
- U.S. Census Bureau . 2000. Projections of the total resident population by five-year age groups and sex, with special age categories: Middle series 2025-2045 . Washington : Census Bureau , Populations Projections Prog., Population Division. U.S. Census Bureau. 2000. Projections of the total resident population by five-year age groups and sex, with special age categories: Middle series 2025-2045. Washington: Census Bureau, Populations Projections Prog., Population Division.
Cited by
Sorted by citations
Health empowerment through activity trackers: An empirical smart wristband study
Nelson, E. C., Verhagen, T., & Noordzij, M. L. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.03.065 · Cited by 164
Chinn, D., & Homeyard, C. (2016) · Health Expectations · doi:10.1111/hex.12520 · Cited by 165
Putting it Simply: a Context-Aware Approach to Lexical Simplification
Biran, O., Brody, S., & Elhadad, N. (2011) · doi:10.7916/d8cv4s20 · Cited by 142
Self-citationRello, L., Baeza-Yates, R. A., Dempere‐Marco, L., & Saggion, H. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40498-6_15 · Cited by 134
Text simplification for children
Belder, J. D., & Moens, M. (2010) · Lirias · Cited by 126
Rello, L., Baeza-Yates, R. A., Bott, S., & Saggion, H. (2013) · International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/2461121.2461126 · Cited by 111
Saggion, H., Štajner, S., Bott, S., Mille, S., Rello, L., & Drndarević, B. (2015) · ACM Transactions on Accessible Computing · doi:10.1145/2738046 · Cited by 121
Reading on the Computer Screen: Does Font Type has Effects on Web Text Readability?
Ali, A. Z. M., Wahid, R., Samsudin, K., & Idris, M. Z. (2013) · International Education Studies · doi:10.5539/ies.v6n3p26 · Cited by 69
A Systematic Literature Review of Research-Derived Touchscreen Design Guidelines for Older Adults
Nurgalieva, L., Jara Laconich, J. J., Baez, M., Casati, F., & Marchese, M. (2019) · IEEE Access · doi:10.1109/access.2019.2898467 · Cited by 61
“, motor impairments [13], [14], [17], [57], [58], cognitive declines [59]–[61], and vision loss [50], [62].”
BackgroundResearch and development in information retrieval
(1985) · Information & Management · doi:10.1016/0378-7206(85)90069-2 · Cited by 57
SymbolChat: A flexible picture-based communication platform for users with intellectual disabilities
Keskinen, T., Heimonen, T., Turunen, M., Rajaniemi, J.-P., & Kauppinen, S. (2012) · Interacting with Computers · doi:10.1016/j.intcom.2012.06.003 · Cited by 50
Berke, L., Kafle, S., & Huenerfauth, M. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173665 · Cited by 44
Self-citationPerusal of readability with focus on web content understandability
Ojha, P. K., Ismail, A., & Kuppusamy, K. S. (2018) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2018.03.007 · Cited by 40
Automatic readability assessment
Huenerfauth, M., & Feng, L. (2010) · Cited by 36
Self-citationCASSA: A Context-Aware Synonym Simplification Algorithm
Baeza-Yates, R. A., Rello, L., & Dembowski, J. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1156 · Cited by 31
Research and development in information retrieval
Serdyukov, P., Hiemstra, D., Preface, I. R., & Castillo, C. (1984) · Computer Compacts · doi:10.1016/0167-7136(84)90079-9 · Cited by 19
The Impact of Lexical Simplification by Verbal Paraphrases for People with and without Dyslexia
Rello, L., Baeza-Yates, R. A., & Saggion, H. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37256-8_41 · Cited by 17
Alonzo, O., Trussell, J., Dingman, B., & Huenerfauth, M. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445038 · Cited by 16
Self-citationAccessibility-Related Publication Distribution in HCI Based on a Meta-Analysis
Colley, M., Kränzle, T., & Rukzio, E. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519701 · Cited by 16
Measuring the readability of medical research journal abstracts
Severance, S., & Cohen, K. B. (2015) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/w15-3815 · Cited by 15
Evaluating the Readability of Text Simplification Output for Readers with Cognitive Disabilities
Yaneva, V., Temnikova, I. P., & Mitkov, R. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4a7fdf4xbeze · Cited by 14
Alonzo, O., & Hassan, S. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746355 · Cited by 13
Säuberli, A., Holzknecht, F., Haller, P., Deilen, S., Schiffl, L. F., Hansen‐Schirra, S., & Ebling, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642570 · Cited by 11
Perspectives of Neurodiverse Participants in Interactive Information Retrieval
Sitbon, L., Berget, G., & Brereton, M. F. (2023) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000086 · Cited by 11
Säuberli, A., Hansen‐Schirra, S., Holzknecht, F., Gutermuth, S., Deilen, S., Schiffl, L. F., & Ebling, S. (2023) · Frontiers in Communication · doi:10.3389/fcomm.2023.1175625 · Cited by 7
Six Good Predictors of Autistic Text Comprehension
Yaneva, V., & Evans, R. (2015) · International conference Recent advances in natural language processing · Cited by 7
Towards accessible search systems
Serdyukov, P., Hiemstra, D., & Ruthven, I. G. (2011) · ACM SIGIR Forum · doi:10.1145/1924475.1924482 · Cited by 5
Measuring Text Comprehension for People with Reading Difficulties Using a Mobile Application
Säuberli, A. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3476474 · Cited by 5
Reading difficulty in adults with intellectual disabilities
Jansche, M., Feng, L., & Huenerfauth, M. (2010) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1878803.1878869 · Cited by 5
Self-citationDittrich, S., Weiß, Z., Schröter, H., & Meurers, D. (2019) · Cited by 4
Automatic Adaptation of Swedish Text for Increased Inclusion
Rennes, E. (2022) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789179292706 · Cited by 6
Weiß, Z., Dittrich, S., & Meurers, D. (2018) · Cited by 3
(2015) · doi:10.18653/v1/w15-38 · Cited by 1
Metadata
- Authors
- Huenerfauth, M., Feng, L., & Elhadad, N.
- Affiliations
- Queens College, CUNY (US)
- City University of New York (US)
- Columbia University (US)
- Venue
- Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility
- Publisher
- ACM
- Published
- 2009-10-25
- Pages
- 3-10
- OpenAlex
- W2129866217
- Cited by
- 35
- References
- 32
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 2 · 2022: 2 · 2021: 2 · 2019: 2 · 2018: 5 · 2016: 3 · 2015: 5 · 2013: 4 · 2012: 1
- FWCI
- 1.41
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) (publishedVersion)
- Funders
- Queens College, City University of New York
- Topics
- Social Sciences › Linguistics and Language › Text Readability and Simplification
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- readability, intellectual disability, text simplification, readability assessment, comprehension assessment, question format, multiple-choice questions, visual aids, news articles, text-to-speech, usability evaluation, adults with disabilities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
