CoChat
Sign Up
Cited by 0Hearing Impairment and CommunicationSubtitles and Audiovisual MediaHearing Loss and Rehabilitation

Effects of Text Release Features on Communication and Problem-Solving in Teams with a Deaf/Hard of Hearing and a Hearing Member Using Automatic Speech Recognition

Open access
Stinson, Michael, L Carpenter, Rebecca, Easton, Donna·2025·Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility
DOI 10.1145/3663547.3759694
Open this paper in CoChat — free accountOpen PDF

Abstract

This study examined effects of three text release features used with Automatic Speech Recognition (ASR)-based automatic captioning and of hearing status on quality of performance on problem-solving tasks, number of words two team members exchanged with each other, and on member's judgments regarding the extent the ASR generated text contained errors. Each of 36 teams, consisting of one deaf/hard of hearing (DHH) and one hearing member, worked online to complete three mapping tasks. These mapping tasks required naming of map locations that were marked by numbers along a pathway indicated by a line with numerous turns. For each of three mapping tasks, the hearing member used a different ASR-based text release feature to produce captions for the DHH member to read, either: (a) controlled release; (b) automatic release with no editing; or (c) automatic release with editing. Results suggested that for DHH and hearing members to solve a problem using automatic captioning with ASR synchronously, it is important for the captioning technology to facilitate rapid exchange of words between members and, at the same time, facilitate dealing with the errors produced by the ASR. Consistent with this proposition, problem-solving performance was highest under the automatic release with editing feature.

AI snapshot

Written from the paper’s full text.

TL;DR

This study compared controlled release, automatic release, and automatic release with editing for ASR-based captioning in 36 teams with one deaf/hard of hearing (DHH) and one hearing member solving online mapping tasks. Automatic release with editing produced the highest problem-solving performance, while automatic release without editing produced the most words.

Key findings

  • Problem-solving performance was highest with automatic release with editing (M = 11.36 out of 15), second highest with automatic release (M = 10.40), and lowest with controlled release (M = 9.89), F(1.99, 139.59) = 3.87, p < .05, partial η² = .05; follow-up tests showed significantly higher performance with automatic release with editing than with controlled release.
  • Members produced the most words with automatic release (M = 215.61), then automatic release with editing (M = 193.54), then controlled release (M = 166.43), F(1.94, 135.56) = 17.07, p < .001, partial η² = .19; hearing members produced more words (M = 251.39) than DHH members (M = 166.43), F(1, 70) = 55.07, p < .001, partial η² = .44.
  • Reported caption errors were most frequent for automatic release with editing (M = 3.49 on a 5-point scale), then automatic release (M = 3.36), then controlled release (M = 3.03), F(1.83, 128.09) = 6.69, p < .01, partial η² = .09; DHH members assigned higher error ratings for both automatic release features than for controlled release.
  • The mean ASR word error rate was 11%, meaning slightly more than every 10th word in captions was an error.

Methods

  • 36 teams, each with one DHH and one hearing member, worked online to complete three mapping tasks adapted from the Montclair mapping task; participants were 72 undergraduate and graduate students at a northeastern U.S. university (36 DHH, 36 hearing).
  • For each task, the hearing member used a different text release feature—controlled release, automatic release with no editing, or automatic release with editing—with order counterbalanced; each task required labeling 15 numbered map locations, and each member had 10 landmark names, including five unique to that member.
  • Measures were number of words produced, number of map locations correctly identified out of 15, and 5-point ratings of how often captions made an error; data were analyzed with mixed 3 (Release Feature) × 2 (Hearing Status) ANOVAs using Greenhouse-Geisser correction, with follow-up paired t-tests and Bonferroni correction.

Limitations

  • The paper does not state limitations.
  • Evident from the text: the sample was 72 undergraduate and graduate students at one northeastern U.S. university, recruited by posters, contacts, and social media, so findings may not generalize to other populations or settings.
  • Evident from the text: the study used adapted online mapping tasks rather than naturally occurring workplace or academic conversations, and the ASR had an 11% mean word error rate, which may limit applicability to other ASR systems or communication contexts.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3663547.3759694

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The interaction effect on error judgments was due to the text release feature affecting identification of errors for DHH members but not for hearing members.

    “Post hoc comparisons indicated that interaction effect was due to the text release feature affecting identification of errors for DHH members but not hearing ones.”
    Quote in the paperSubstantial claim
  • Findingreasoning

    DHH members reported noticing more errors with both automatic release features than with controlled release.

    “In addition, DHH members reported noticing more errors with both automatic release features than with controlled release.”
    Quote in the paperSubstantial claim
  • Findingown data

    The interaction between Release Feature and Hearing Status on judgments of caption errors was significant, F(1.83, 128.09) = 9.47, p < .001, partial eta squared = .12.

    “The interaction between Release Feature and Hearing Status was also significant, F (1.83, 128.09) = 9.47, p < .001,p 𝜂 2 = .12(medium effect).”
    Quote in the paperSubstantial claim
  • Findingown data

    Hearing members (M = 251.39) produced more words than did DHH members (M = 166.43), F(1, 70) = 55.07, p < .001, partial eta squared = .44.

    “In addition, hearing members (M = 251.39)produced more words than did DHH ones (M = 166.43),F (1, 70) = 55.07,p < .001,p 𝜂 2 = .44(large effect).”
    Quote in the paperNot a substantial claim
  • Findingown data

    DHH members assigned higher mean ratings for occurrence of errors for both automatic release with editing and automatic release than for controlled release.

    “DHH members assigned higher mean ratings for occurrence of errors for both automatic release with editing, and for automatic release than for controlled release.”
    Quote in the paperSubstantial claim
  • May combine two claims. 12
  • Classified as probably not a substantial claim. 8

  • May combine two claims.

    Findingown data

    Members produced the most words with automatic release (M = 215.61), the second most with automatic release with editing (M = 193.54), and the third most with controlled release (M = 166.43), F(1.94, 135.56) = 17.07, p < .001, partial eta squared = .19.

    “The ANOVA for production of text indicated that members produced the most words with automatic release (M = 215.61), the second most with automatic release with editing (M = 193.54),and the third most with controlled release (M = 166.43),F (1.94, 135.56) = 17.07, p < .001,p 𝜂 2 = .19(large effect).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    DHH members reported noticing more errors with automatic release with editing than with automatic release without editing, although that difference was not statistically significant.

    “DHH members reported noticing more errors with automatic release with editing than with automatic release without editing, although that difference was not statistically significant.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    When hearing members produced text using ASR, the mean word error rate was 11%, i.e., slightly more than every 10th word in the captions members read was an error.

    “When hearing members produced text by using ASR, the mean word error rate was 11%.”
    • Also stated in: § 4 Discussion · plain
      “Thus, slightly more than every 10th word in the captions that members read was an error.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Members reported that caption errors occurred most often for automatic release with editing (M = 3.49), second most often for automatic release (M = 3.36), and least often for controlled release (M = 3.03), F(1.83, 128.09) = 6.69, p < .01, partial eta squared = .09.

    “Members reported that errors occurred most often for automatic release with editing (M = 3.49, 5-point scale), second most often for automatic release (M = 3.36), and least often for controlled release (M = 3.03), F (1.83, 128.09) = 6.69, p < .01,p 𝜂 2 = .09(medium effect).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Neither the effect of Hearing Status nor the interaction between Release Feature and Hearing Status on problem-solving performance was statistically significant.

    “Neither the effect of Hearing Status, F (1, 70) = .87,nor the interaction between Release Feature and Hearing Status, F (1.99, 139.59) = .295,were statistically significant.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Problem-solving performance was highest with automatic release with editing (M = 11.36 out of 15), second highest with automatic release (M = 10.40), and lowest with controlled release (M = 9.89), F(1.99, 139.59) = 3.87, p < .05, partial eta squared = .05.

    “An ANOVA revealed that performance was highest with automatic release with editing (M = 11.36;out of 15 possible points), second highest with automatic release (M = 10.40), and lowest with controlled release (M = 9.89), F (1.99, 139.59) = 3.87, p < .05,p 𝜂 2 = .05(small effect).”
    • Also stated in: § 4 Discussion · plain
      “Consistent with this proposition, problem solving performance was highest with automatic release with editing.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Participation by DHH individuals in group conversations is critical for workplace and academic situations.

    “Participation by DHH individuals in such conversations is critical for the workplace and academic situations [1][2][3][4][5][6][7].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [1]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [2]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [3]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Automatic speech recognition may facilitate interactions between DHH and hearing individuals, especially in one-on-one and small-group real-time synchronous conversations, because ASR-based captions may give DHH individuals access to what hearing people are saying.

    “Automatic speech recognition (ASR) may have potential in facilitating interactions between DHH individuals and hearing ones, especially in one-on-one and small group real-time (i.e.synchronous) conversations that arise spontaneously because ASR-based captions may provide DHH individuals access to what hearing people are saying [8,9].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [8]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    Automatic release without editing has the fewest steps, automatic release with editing adds the step of correcting errors, and controlled release has the most steps, requiring separate steps to activate the ASR and to release text.

    “Automatic release without editing has the fewest steps.Automatic release with editing includes the step of correcting errors when the hearing member determines a word error has occurred that needs correction.Controlled release has the most steps, requiring separate steps for users to activate the ASR and to release text.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The order of the three text release features was counterbalanced across the mapping tasks.

    “The order of text release features was counterbalanced.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    If ASR produces word errors that misrepresent intended communication, it is critical for the sender to be able to easily and quickly correct the errors.

    “If ASR produces word errors that misrepresent intended communication, it is critical for the sender to be able to easily and quickly correct the errors [4,[16][17][18][19][20].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [4]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [16]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [17]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Performance on the map location tasks should in part reflect the quality of communication between the two teammates.

    “Performance on the map location tasks should in part reflect the quality of communication between the two teammates.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    DHH members could have been more confused about the intended message when hearing members used automatic release without editing than when they used automatic release with editing.

    “Thus, DHH members could have been more confused about the intended message when hearing members used automatic release without editing than when they used automatic release with editing.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Knowing the correct word could have reduced confusion in communication and increased the likelihood that correct sharing of knowledge between teammates occurred.

    “Knowing the correct word could have reduced confusion in communication and increased the likelihood that correct sharing of knowledge between teammates occurred.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    For DHH and hearing members to solve a problem using ASR-based automatic captioning synchronously, it is important for the technology to facilitate rapid exchange of words between members and, at the same time, to facilitate dealing with the errors produced by the ASR.

    “Results suggested that for DHH and hearing members to solve a problem using automatic captioning with ASR synchronously, it is important for the captioning technology to facilitate rapid exchange of words between members and, at the same time, facilitate dealing with the errors produced by the ASR.”
    • Also stated in: § 4 Discussion · hedged
      “Results suggested that for DHH and hearing members to solve a problem by communicating with ASR-based automatic captioning, it is important for the technology to facilitate exchange of words between members and, at the same time, expedite dealing with the errors produced by the ASR [25].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [25]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Hearing members may have corrected errors before sending text with controlled release, and thus DHH teammates were less likely to notice the errors.

    “Hearing members may have corrected errors before sending text with controlled release, and thus DHH teammates were less likely to notice the errors.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The editing display information may have helped DHH members recognize that an error had occurred and identify the correct, intended word.

    “This information may have helped DHH members recognize that an error had occurred and identify the correct, intended word.”
    Quote in the paperSubstantial claim
17 of 17

  • The paper relies on this without support.

    Publicly available technologies that integrate ASR with automatic release do not permit editing of text that has been previously released.

    “To our knowledge publicly available technologies that integrate ASR with automatic release do not permit editing of text that has been previously released.”
1 of 1

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(1.94, 135.56) = 17.07reported p < .001 · recomputed p < .001

    “F (1.94, 135.56) = 17.07, p < .001”

  • ConsistentF(1, 70) = 55.07reported p < .001 · recomputed p < .001

    “F (1, 70) = 55.07,p < .001”

  • ConsistentF(1.94, 135.56) = 16.65reported p < .001 · recomputed p < .001

    “F (1.94, 135.56) = 16.65,p < .001”

  • ConsistentF(1.99, 139.59) = 3.87reported p < .05 · recomputed p = .023

    “F (1.99, 139.59) = 3.87, p < .05”

  • ConsistentF(1.83, 128.09) = 6.69reported p < .01 · recomputed p = .002

    “F (1.83, 128.09) = 6.69, p < .01”

  • ConsistentF(1.83, 128.09) = 9.47reported p < .001 · recomputed p < .001

    “F (1.83, 128.09) = 9.47, p < .001”

6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Word-production means imply different overall averages

    Major

    Because the same 72 participants contributed to all three release-feature conditions, averaging the three feature means should give the same overall mean as averaging the two equally sized hearing-status groups. The feature means average 191.86 words, while the hearing-status means average 208.91 words.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. 10.7326/0003-4819-140-5-20040302000011 · DOI
  2. Ronald R. Kelly, Andrew Quagliata, Richard DeMartino, Victor Perotti. 2016. 21st-century deaf workers: Going beyond “just employed” to career growth and entrepreneurship. In Mark Marschark, Venetta Lampropoulou, and Emmanouil Skordilis (Eds.) Diversity in Deaf Education. pp. 473-505. New York: Oxford University Press.
  3. Pam Millet. 2021. Accuracy of speech-to-text captioning for students who are deaf or hard of hearing. Journal of Educational Pediatric, and (Re)Habilitative Audiology, 25, 2021-2022.

Authors
Stinson, M., L Carpenter, R., & Easton, D.
Affiliations
  • Rochester Institute of Technology (US)
  • National Technical Institute for the Deaf, Rochester Institute of Technology, Rochester, New York, USA
ORCID
Venue
Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility
Publisher
ACM
Published
2025-10-22
Pages
1-4
DOI
10.1145/3663547.3759694
OpenAlex
W4415444970
Semantic Scholar
73acd68acb8b24fe8862e864bed138c8fe162be1
Cited by
0
References
22
FWCI
0.00
Type
conference paper
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor, 365 d)
Locations
International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) (publishedVersion)
Funders
National Science Foundation (1954284)
Topics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Hearing Impairment and Communication
  • Social Sciences › Arts and Humanities › Language and Linguistics › Subtitles and Audiovisual Media
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Hearing Loss and Rehabilitation
Keywords
deaf and hard of hearing, speech recognition, automatic captioning, team communication, problem-solving performance, online collaboration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-20.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-20.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review