CoChat
Sign Up
Cited by 59Digital Accessibility for DisabilitiesUsability and User Interface DesignTactile and Sensory Interactions

Understanding Blind Users' Web Accessibility and Usability Problems

Open access
The University of North Carolina - Greensboro, Babu, Rakesh, Singh, Rahul, The University of North Carolina at Greensboro, USA, Ganesh, Jai, Infosys Technologies Limited·2010·AIS Transactions on Human-Computer Interaction
DOI 10.17705/1thci.00015
Open this paper in CoChat — free accountOpen PDF

Abstract

Our motivation for this research is the belief that blind users cannot participate effectively in routine Web-based activities due to the lack of Web accessibility and usability for non-visual interaction. We take a cognitive, user-centered, task-oriented approach to develop an understanding of accessibility and usability problems that blind users face in Web interactions. This understanding is critically needed to determine accessibility and usability requirements for non-visual Web interaction. We employ verbal protocol analysis for an in-depth examination of difficulties participants face in completing an online assessment through a course management system. We analyze the problems that hinder accessibility and usability and explain the nature of these problems in terms of design principles. Our study contributes an effective method for qualitative evaluation of Web accessibility and usability. Our findings will guide future research to develop more accessible and usable Web applications for blind users.

Integrity signals

Registry

Crossref · 10.17705/1thci.00015

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 89 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Speech output from the screen-reader provided clues about actions participants took that they did not verbalize.

    “Speech output of the screen-reader provided valuable clues to what actions participants took that they did not verbalize.”
    Quote in the paperSubstantial claim
  • Findingown data

    Using coded verbalizations and screen-reader speech proved useful for capturing the complete interaction process and tracing additional problems.

    “This technique proved useful in capturing the complete interaction process and tracing additional problems users experienced.”
    Quote in the paperSubstantial claim
  • Findingown data

    It took time and exploration for participants to figure out that they needed to type their input rather than select an answer as in multiple-option questions.

    “It took some time and exploration for these participants to figure out that they needed to type in their input and not select an answer as in multiple-option questions.”
    Quote in the paperSubstantial claim
  • Findingown data

    Qualitative analysis revealed that blind users are likely to face four major accessibility and usability problems while completing online assessments.

    “Our qualitative analysis reveals that blind users are likely to face four major accessibility and usability problems while completing online assessments.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The six participants' experiences are representative enough of blind users to support general conclusions about blind users' Web accessibility and usability problems.
  • Findingown data

    Participants had difficulty understanding how to respond to short-answer questions.

    “We observed that our participants had difficulty understanding how to respond to short-answer questions.”
    • Also stated in: § Ambiguity about responding to short-answer questions · plain
      “Our participants had difficulty understanding how to provide a response to the short-answer question.”
    Quote in the paperSubstantial claim
  • May combine two claims. 42
  • Classified as probably not a substantial claim. 47
  • The quote only partly states this claim. 5

  • May combine two claims.

    Findingown data

    Participants were vulnerable to being unaware of inadvertently skipping assessment questions, and blind users in current LMS are prone to missing questions without noticing.

    “Our analysis tells us that participants were vulnerable to being unaware of inadvertently skipping assessment questions.”
    • Also stated in: § Ambiguity about responding to short-answer questions · plain
      “One of our findings is that in current LMS, blind users are prone to missing questions without even noticing.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    No participant reported having used an LMS such as Blackboard, Moodle, or WebCT.

    “No one reported having used an LMS (e.g.Blackboard, Moodle or WebCT).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participants encountered an object labeled Graphic Question 3 Answers just before the formatting controls, which seemed to imply that possible answers followed.

    “Participants came across an object labeled “Graphic Question 3 Answers” just before these controls.This seemed to imply that what followed was a set of possible answers.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Categorizing segments as dissonance and failures helped trace problems blind subjects encountered during interaction.

    “Segments categorized as dissonance and failures helped us trace problems of blind subjects during the interaction process.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The user believed she could move to the next question by hitting Enter on a checkbox, but the system did not lead her forward.

    “The user believed she could move to the next question by hitting Enter on a checkbox, but the system did not lead her forward.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participant S6, while typing an essay-type answer, pressed Backspace to delete a typographical error, became confused by unexpected screen-reader announcements, and later realized he had been expelled from the assessment and was not allowed to retake it.

    “Here, participant S6 is responding to the essay-type question by typing an answer in the input field.He hits the Backspace key to delete a typographical error.However, he is confused by the screen-reader's unexpected announcements.Later, he realizes that the page had changed; he was expelled from the assessment and not allowed to retake it.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participant S4's experience illustrated the inconsistency: after successfully using Enter to move from question 1 to question 2, she pressed Enter twice on a question 2 option to move to question 3, but the system returned her to question 1, causing confusion and the belief that she had made an error.

    “In Question 2, she hit Enter twice on an option with the goal of moving to Question 3. Instead, the system took her back to Question 1.She became confused, believing she must have committed an error.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participant S4 was vulnerable to skipping a question but avoided the error after being prompted to verify her location.

    “Table 2 also presents evidence of S4's vulnerability to the error, which she avoided after being prompted to verify her location.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    The labels on the formatting controls do not effectively communicate their purpose, causing blind users to feel disoriented.

    “The labels on these formatting controls do not communicate their purpose effectively, so the user feels disoriented.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Blind users feel frustrated and dissatisfied when the system expels them from the assessment without allowing a retake after they use Backspace.

    “It expels the user from the assessment without allowing a retake.The user feels frustrated and dissatisfied.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The study's evidence indicates that blind users prefer input areas that use an edit field, which is common on Web pages and easy to operate with a screen reader.

    “Our evidence also points out that blind users prefer input areas that use an edit field.This type of input field is most common in Web pages, and is easy to operate with a screen-reader.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When a participant used the backspace key to delete text in the input area of an essay-type question, the system expelled him completely out of the assessment and did not allow a second attempt.

    “We observed that when a participant used the backspace key to delete text in the input area of an essay-type question, the system expelled him completely out of the assessment, and did not allow a second attempt.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    In the assessment system, a user could also submit a response to a multiple-option question by pressing Enter twice on the radio button or checkbox for an option.

    “Users can also submit their response if they hit Enter twice on the radio button or the checkbox corresponding to an option.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participants ignored messages such as 'Question 2 of 3' and 'This question may be incomplete' because the messages were announced without context and did not adequately capture their attention.

    “These messages are announced without the context, and therefore do not adequately capture participants' attention.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The Enter-twice method of submitting a multiple-option response did not always advance to the next question page; sometimes it returned the user to the previous question page.

    “However, this second method of trying to submit the response does not always result in moving to the next question page.Users may sometimes go back to the previous question page instead of moving forward.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participants in the study were unaware when they committed errors, and in those situations they had no chance to recover.

    “Table 3 presents evidence that demonstrates how participants were oblivious to committing errors.In such situations, the user lost all possibilities of recovery.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Participant S1 skipped question 2 without answering it by activating the 'next question' button after the screen reader announced successful submission of question 1 and positioned him near that button on the question 2 page.

    “It shows that S1 activated this button, completely unaware that he skipped question 2.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Because Web interaction is sequential, a blind user perceives only a small piece of content at a time and loses contextual information.

    “The sequential nature of Web interaction means that at any given point, the user perceives only a snippet of the content, losing all contextual information (Lazar et al., 2007).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lazar et al., 2007 source

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    This calls for a close examination of the user-system interaction process.

    “This requires a close examination of the user-system interaction process (Zhang et al., 1999).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhang et al., 1999 source

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Web interaction, like other human behavior, involves perception, cognition, and action.

    “Web interaction, like other human behavior, involves three types of basic processes: perception, cognition, and action.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    For user-system interaction to be effective, both technical accessibility and cognitive usability are needed.

    “Effective user-system interaction requires both technical accessibility and cognitive usability (Norman, 1988).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Norman, 1988

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    Problems occur when Web design makes users expend additional physical and mental effort in perception, cognition, and action.

    “Problems arise when Web design forces the user to spend extra physical and mental effort in these processes (Foley et al., 1984).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Foley et al., 1984 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Screen-readers vocalize Web page content in a top-left to bottom-right order.

    “Screen-readers read information presented on a Web page aloud from top left to bottom right (Leuthold et al., 2008).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Leuthold et al., 2008 source

  • May combine two claims.

    Interpretationreasoning

    The literature neither explains where and why blind users encounter Web interaction problems nor offers insight into their cognition and behavior when problems occur.

    “It does not explain where and why blind users face problems during Web interaction, nor does it provide insight into blind user's cognition and behavior under a problem situation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    The existing literature acknowledges that blind users' Web interaction problems have multiple dimensions, yet it fails to clearly explain what those problems are like.

    “Extant literature recognizes that blind users' Web interaction problems are multi-dimensional, but does not clearly explain their nature.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Evaluating system usability requires knowing how users think, perceive, and act while interacting.

    “To evaluate system usability, we need to know how users think, perceive, and act during their interaction (Norman, 2001).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Norman, 2001

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Blind people face challenges interacting with Web sites and applications, primarily because current Web technology lacks accessibility and usability.

    “For the 45 million blind people around the world, interacting with Web sites and Web applications is a challenging task (Theofanos and Redish, 2003;Erin, 2006;Brophy and Craven, 2007;Lazar et al., 2007).This is primarily due to a lack of accessibility and usability in current Web technology (Gerber and Kirchner, 2001;Theofanos and Redish, 2003;Di Blas et al. 2004;Correani et al., 2004;Leuthold et”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Theofanos and Redish, 2003

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Erin, 2006

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Brophy and Craven, 2007 source

  • The quote only partly states this claim.

    Interpretationreasoning

    These screen-reader limitations make it hard for blind users to understand the conveyed information.

    “These shortcomings make it difficult for the user to understand the information being conveyed.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Blind users cannot quickly scan a page, which makes it hard for them to find information relevant to their goal.

    “Inability to quickly scan a page means that the user has trouble locating goal-relevant information on the Web (Di Blas, et al, 2004).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Di Blas, et al, 2004

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    Accurately understanding blind users' accessibility and usability problems, and thus developing effective solutions, requires understanding their Web experiences.

    “Without understanding blind users' Web experiences, we cannot accurately understand the nature of their accessibility and usability problems, and therefore cannot develop effective solutions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    These distinctive traits of blind users make the overall findings of prior studies questionable and indicate a need to examine their cognition and behavior more closely during Web interaction.

    “These unique characteristics of blind users render the overall findings of studies questionable and point to a need for a closer examination of their cognition and behavior during web interaction.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    If a system is inaccessible, it cannot be usable; however, making a system accessible does not by itself make it usable.

    “A system that is not accessible is not usable; however an accessible system does not guarantee usability (Di Blas et al., 2004).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Di Blas et al., 2004

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    The coding scheme was adapted from Norman's seven-stage action model.

    “We developed a coding scheme, adapted from Norman's (1988) seven-staged action model, to characterize the cognition and behavior of blind users' Web interaction.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Norman, 1988

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    The seven-staged action model is useful for examining the full interaction between a user and a system.

    “The seven-staged action model helps us examine the complete interaction between a user and a system (Norman, 1988).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Norman, 1988

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    They give us a foundation for understanding blind users' cognition and behavior and for characterizing their Web interaction problems.

    “They provide a basis for us to understand the cognition and behavior of blind users and characterize the nature of problems that they face in web interaction.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Usability principles developed for sighted users are applicable to blind users.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Failure to accomplish a goal indicates a gulf of execution.

    “Failure to accomplish goal indicates gulf of execution (Norman, 1988).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Norman, 1988

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Cognitive dissonance grows with a decision's importance, impact, and difficulty of reversal.

    “Cognitive dissonance increases with the importance and impact of a decision, along with the difficulty of reversing it (Aronson, 1969).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Aronson, 1969 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    WCAG recommendations are the main guidance for developers and designers on accessible Web design.

    “Since then, recommendations of WCAG 1.0, updated to WCAG 2.0 in December 2008 (Trace Center, 2008), represent the primary source of guidance for developers and designers on accessible Web design (Kelly et al., 2005).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kelly et al., 2005 source

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Problems arise when people cannot map their intentions to interpretations or their physical actions to system states.

    “In other words, the difficulty is solely because they fail to map intentions to interpretations and physical actions to system states.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Norman, 1988

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    This model accounts for both cognitive and physical activities.

    “This approximate psychological model accounts for both cognitive and physical activities (Zhang et al., 1999).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Zhang et al., 1999 source

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    These principles were created for sighted users and represent the best available knowledge about accessible and usable interface design.

    “These principles were developed for accessibility and usability of systems for sighted user interaction and represent our best knowledge of accessible and usable systems interface design.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Usability principles developed for sighted users are applicable to blind users.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Consistency in action-outcome mapping allows blind users to focus on the online task without worrying about system functionality and navigational issues.

    “Consistency in the action-outcome mapping allows blind users to focus on accomplishing the objective of the online task without worrying about system functionality and navigational issues.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Norman's seven-staged action model describes user-system interaction and identifies conditions that cause usability problems.

    “Norman (1988) proposed the seven-staged action model to explain the interaction process between a user and a system, and identify conditions that give rise to usability problems.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Norman, 1988

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisenone given

    WCAG sets out four principles of Web accessibility.

    “The Web Content Accessibility Guidelines (WCAG) provides four principles of Web accessibility.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Perceivability recommendations call for modifying graphical interfaces to help screen-readers access them.

    “For instance, recommendations on perceivability prescribe modifying graphical interface to facilitate screen-reader access (Leuthold et al., 2008).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Leuthold et al., 2008 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Dissonance also drives us to change future actions and behavior to restore consistency.

    “This dissonance also motivates us to change our future actions and behavior for achieving consonance (Aronson, 1969).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Aronson, 1969 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The study included six blind participants with an average age of 23 years.

    “Participants in this study included six blind individuals with an average age of 23 years.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    People naturally seek consistency among their beliefs, feelings, opinions, values, and environmental knowledge.

    “By nature, we tend to seek consonance among various cognitions, including beliefs, affect, opinion, values, and knowledge of the environment (Festinger, 1957).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Festinger, 1957 source

  • Classified as probably not a substantial claim.

    Premisecitation

    Dissonance in interpretation indicates a gulf of evaluation.

    “Each form of dissonance indicates gulf of evaluation (Norman, 1988).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Norman, 1988

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    At the time of the study, all participants were enrolled in Institutions of Special Education for the Blind.

    “At the time of the study, all participants were enrolled in Institutions of Special Education for the Blind (ISEB).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Some governments have written WCAG recommendations into Web accessibility laws.

    “Several governments have incorporated WCAG recommendations into laws on Web accessibility (e.g.Section 508 of the U.S. federal government) (Leuthold et al., 2008).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Leuthold et al., 2008 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Blackboard is the most popular LMS and is used by more than 3,700 academic institutions across 60 countries.

    “Blackboard, the most popular LMS, is used by more than 3,700 academic institutions across 60 countries (http://en.wikipedia.org/wiki/Blackboard_Inc).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites http://en.wikipedia.org/wiki/Blackboard_Inc

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation

    WCAG is necessary but not sufficient for evaluating Web accessibility for blind users.

    “Our literature analysis informs us that WCAG is necessary but not sufficient for evaluation of Web accessibility for the blind.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: WCAG compliance is a good starting point for achieving Web accessibility for the blind.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    If dissonance becomes strong enough to overcome a cognition's resistance, we alter or remove that cognition.

    “When it exceeds the threshold to overcome the resistance of a particular cognition, we change or eliminate that cognition (Festinger, 1957).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Festinger, 1957 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Dissonance occurs because a new idea must be accommodated.

    “Dissonance arises due to a need for accommodating a new idea.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The current WCAG guidelines contain four guidelines and 18 checkpoints arranged hierarchically.

    “The current version of WCAG guidelines (World Wide Web Consortium, 2008) includes a hierarchy of four guidelines and 18 checkpoints.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites World Wide Web Consortium, 2008

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    WCAG is the de facto standard for Web accessibility.

    “The WCAG is the de facto standard on Web accessibility.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Verbal protocol analysis is a direct observation method in which subjects think aloud while performing a task.

    “Our method includes verbal protocol analysis (VPA), a kind of direct observation in which subjects think aloud while performing a task (Ericsson and Simon, 1984).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Ericsson and Simon, 1984

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    This motivates the user to change either her cognition or her behavior to reduce dissonance.

    “This motivates her to change either her cognition or her behavior (Festinger, 1957) to reduce dissonance.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Festinger, 1957 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Both the gulf of execution and the gulf of evaluation involve roadblocks and extra cognitive effort, which hinder goal accomplishment.

    “The gulfs of execution and evaluation are both associated with roadblocks and extra cognitive effort, making it difficult for the user to accomplish her goal.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Norman, 1988

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The user forms an expectation about a system's behavior in response to the method chosen.

    “Our contention is that the user forms an expectation about a system's behavior in response to the method chosen.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Johnson-Laird, 1989 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    The four guidelines map to four accessibility principles: perceivability, operability, understandability, and robustness.

    “The four guidelines correspond to four principles of Web accessibility: perceivability, operability, understandability, and robustness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    All participants had at least five years of computer and Web experience using the Jaws screen-reader, and none had ever used computers with sight.

    “All participants had at least 5 years experience using computers and the Web through Jaws screen-reader, and none had ever interacted with computers using sight.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Many experts question whether WCAG recommendations accurately capture blind users' accessibility and usability needs.

    “Many experts doubt that WCAG recommendations accurately represent accessibility and usability needs of blind users (Di Blas et al., 2004;Clark, 2006;Kelly et al., 2005).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Di Blas et al., 2004

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Clark, 2006

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Kelly et al., 2005 source

  • May combine two claims.

    Interpretationreasoning

    The lack of visibility of the input area worsens the blind user's difficulty, causing her to spend much time exploring the page and possibly to forego answering the question if she cannot find the input area.

    “The user's dilemma is confounded by the lack of visibility of the input area.As a result, the user spends much time exploring the page, and may even forego answering the question if she is unable to find the input area.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The finding that participant S1 skipped question 2 was based on analysis of the screen reader's announcement without S1's verbalization.

    “It is noteworthy that this finding was based on analysis of the screen-reader's announcement in the absence of S1's verbalization.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    The short-answer problem violates the principles of efficiency and learnability.

    “This violates the principles of efficiency and learnability (Nielsen, 1993).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Nielsen, 1993

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims. The quote only partly states this claim.

    Interpretationown data

    Because the cursor focus lands on the 'Go to Next Question' button, a blind user may think the previous activation failed and press it again, thereby unintentionally skipping the question.

    “The user is likely to misunderstand that the previous attempt of activating the button failed, and therefore makes another attempt to do so.As a result, she unintentionally skips the question”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The problem of blind users skipping a question is both an accessibility problem and a usability problem.

    “This represents both an accessibility and a usability problem.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The essay-type question ambiguity violates the usability principles of learnability and efficiency.

    “This is a violation of the principles of learnability and efficiency.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Repeated cognitive dissonance across multiple attempts prevents a blind user from understanding the process of answering multiple-option questions, forcing her to relearn the system each time and creating extra steps or increased cognitive load.

    “However, repeated dissonance across multiple attempts will prevent her from understanding the process of answering questions in this format.She must relearn how the system works in every instance of use.This creates problems in the form of extra steps or increased cognitive load (Norman, 2001).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Norman, 2001

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation

    The backspace problem violates Nielsen's satisfaction criterion for usability because participants were dissatisfied with the system's unreliable behavior, especially since they could not complete their task.

    “It is a usability problem as the system violates Nielsen's (1993) satisfaction criteria.Participants were dissatisfied with its unreliable behavior, particularly since they could not complete their task.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Nielsen's (1993)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The findings have implications for design of Web systems used not just for assessments, but also for similar purposes including interactive forms and questionnaires.

    “These findings have implications for design of Web systems used not just for assessments, but also for similar purposes including interactive forms and questionnaires.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The three question formats used (multiple-choice, multiple-answer, and short-answer) are the three most common methods for soliciting responses in Web-based questionnaires and online interactive applications.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation

    The accessibility character of each problem was determined using the authors' conceptualization of perceivability, operability, understandability, and robustness; the usability character was determined using Nielsen's, Norman's, and Shneiderman-Plaisant's good design principles.

    “We determined the accessibility character of a problem [marked by (a)] based on our conceptualization of the principles of perceivability, operability, understandability and robustness.We determined the usability character of a problem [marked as (u)] using Nielsen's (1993), Norman's (1988) and Shneiderman-Plaisant's (2004) good design principles.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Nielsen's (1993)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Norman's (1988)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Shneiderman-Plaisant's (2004)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • The quote only partly states this claim.

    Interpretationcitation

    Under Norman's Action Model, the skipping-question problem constitutes a gulf of evaluation because blind users cannot perceive or interpret information that a new question is available.

    “According to Norman's (1988) Action Model, this indicates a gulf of evaluation because users cannot perceive or interpret the information that a new question is available.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Norman's (1988)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    The multiple-option problem reflects a violation of usability criteria including consistency, memorability, and good Mapping.

    “This problem reflects a violation of usability criteria including consistency, memorability, and good Mapping.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The short-answer problem compromises understandability and violates WCAG's accessibility criteria.

    “It also compromises the understandability factor, thereby violating WCAG's accessibility criteria.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation

    The lack of one-to-one correspondence between user action and system response violates the good mapping principle.

    “This represents a violation of the good mapping principle (Norman, 1988).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Norman, 1988

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The backspace problem is both an accessibility problem and a usability problem under good Web design principles.

    “From the viewpoint of good Web design principles, this is an accessibility and a usability problem.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Ambiguity about the response method for essay-type questions is both an accessibility problem and a usability problem.

    “Ambiguity about response method for essay-type questions is both an accessibility and usability problem.”
    Quote in the paperSubstantial claim
80 of 80

  • The paper relies on this without support.

    Usability principles developed for sighted users are applicable to blind users.

    “These principles were developed for accessibility and usability of systems for sighted user interaction and represent our best knowledge of accessible and usable systems interface design.We adopt and apply these principles to study the accessibility and usability problems that blind users face in Web interaction.”

  • The authors state this as an assumption.

    WCAG compliance is a good starting point for achieving Web accessibility for the blind.

    “Acknowledging its value, along with its limitations, we believe WCAG compliance is a good starting point in achieving Web accessibility for the blind.”

  • The paper relies on this without support.

    Online assessment is a representative Web-based task for blind users.

    “We chose online assessment as the context of our investigation and employed verbal protocol analysis, which is an effective technique for studying users' experience with a system (Todd and Benbasat, 1987).We asked blind participants to complete a representative online assessment while simultaneously verbalizing their thoughts out loud.”

  • The paper relies on this without support.

    The three question formats used (multiple-choice, multiple-answer, and short-answer) are the three most common methods for soliciting responses in Web-based questionnaires and online interactive applications.

    “By including the three types of assessment question formats: multiple-choice, multiple-answer, and short-answer, we have covered the three most common methods for soliciting user responses through Web-based questionnaires and online interactive applications.”

  • The paper relies on this without support.

    The designed online assessment over Blackboard is representative of common Web-based tasks in academic environments.

    “To create a context of Web interaction, we designed a representative online assessment over the Blackboard LMS.”
    “online assessment, which is a common Web-based task in an academic environment.”

  • The paper relies on this without support.

    The six participants' experiences are representative enough of blind users to support general conclusions about blind users' Web accessibility and usability problems.

    “Our qualitative analysis reveals that blind users are likely to face four major accessibility and usability problems while completing online assessments.”

  • The paper relies on this without support.

    Common word-processing practice includes using the backspace key to delete typographical errors, and input fields are expected to support such operations.

    “Use of backspace to delete typographical errors is a common operation in word processing.Input fields are meant to support word processing operations; users will typically use the backspace key to delete text.”
7 of 7

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Citation year differs from the reference list

    Minor

    The in-text citation gives Tonn-Eichstädt's publication year as 2006, while the corresponding reference lists 2009.

  • Example of an evaluation gulf conflicts with its definition

    Minor

    The paper defines the gulf of execution as difficulty translating an intention into an action. It later labels difficulty identifying an action sequence as a gulf of evaluation instead.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 9

  1. [4]4. New York: Academic Press, pp. 1-34
  2. [15]15. Erin, J. N. (2006) "100 Years of Education," Journal of Visual Impairment & Blindness (100) 6, 35-51.
  3. [20]20. Gerber, E. (2002) "Surfing by Ear: Usability Concerns of Computer Users Who are Blind or Visually Impaired," Access World (3) 1, pp. 38-43.
  4. [21]21. Goodhue, D. L. (1986) "IS Attitudes: Towards Theoretical Definition and Measurement Clarity." Paper presented at International Conference on Information Systems: San Diego, CA, December 12-15, 1986.
  5. [44]44. Norman, D. A. (1988) The Design of Everyday Things, 1st edition. New York: Basic Books.
  6. [45]45. Norman, D. A. (2001) The Design of Everyday Things, 4th edition. London: MIT Press.
  7. [47]47. Shneiderman, B. and C. Plaisant (2004) Designing the User Interface, 4th edition. Reading, MA: Addison-Wesley.
  8. [50]50. Theofanos, M. F. and J. Redish (2003) "Guidelines for Accessible and Usable Web Sites: Observing Users Who Work With Screen Readers," Interactions (10) 6, pp. 38-51.
  9. [61]61. Marina del Rey, CA, April 15-17, 1998. Transactions on Human-Computer Interaction ISSN: 1944-3900 Editors-in-Chief http://thci.aisnet.org/

Cited by

Sorted by citations

Related

Authors
The University of North Carolina - Greensboro, Babu, R., Singh, R., The University of North Carolina at Greensboro, U., Ganesh, J., & Limited, I. T.
Affiliations
  • University of North Carolina at Greensboro (US)
  • Infosys (India) (IN)
Corresponding author
Babu, Rakesh
ORCID
Venue
AIS Transactions on Human-Computer Interaction
Publisher
Association for Information Systems
Published
2010-09-30
Volume
2
Issue
3
Pages
73-94
ISSN
1944-3900 (ISSN-L)
DOI
10.17705/1thci.00015
OpenAlex
W2125440472
Cited by
59
References
57
Citations by year
2026: 1 · 2025: 3 · 2024: 5 · 2023: 6 · 2022: 4 · 2021: 4 · 2020: 2 · 2019: 2 · 2018: 6 · 2017: 4 · 2016: 3 · 2015: 2 · 2014: 6 · 2013: 5 · 2012: 5
FWCI
13.74
Type
journal article
Language
en
Open access
Locations
  • AIS Transactions on Human-Computer Interaction (publishedVersion)
  • NC Digital Online Collection of Knowledge and Scholarship (The University of North Carolina at Greensboro) (submittedVersion)
Funders
  • National Science Foundation (0938539)
  • Norges Forskningsråd
Topics
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
Keywords
web accessibility, usability, blind users, non-visual interaction, verbal protocol analysis, virtual learning environments, online assessment, accessibility requirements, design principles, user-centered design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-09-30.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-30.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review