CoChat

An investigation into the effectiveness of using acoustic touch to assist people who are blind

Open access

The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index; 5 of the 37 sources its claims cite were checked for support.

Abstract

Wearable smart glasses are an emerging technology gaining popularity in the assistive technologies industry. Smart glasses aids typically leverage computer vision and other sensory information to translate the wearer's surrounding into computer-synthesized speech. In this work, we explored the potential of a new technique known as "acoustic touch" to provide a wearable spatial audio solution for assisting people who are blind in finding objects. In contrast to traditional systems, this technique uses smart glasses to sonify objects into distinct sound auditory icons when the object enters the device's field of view. We developed a wearable Foveated Audio Device to study the efficacy and usability of using acoustic touch to search, memorize, and reach items. Our evaluation study involved 14 participants, 7 blind or low-visioned and 7 blindfolded sighted (as a control group) participants. We compared the wearable device to two idealized conditions, a verbal clock face description and a sequential audio presentation through external speakers. We found that the wearable device can effectively aid the recognition and reaching of an object. We also observed that the device does not significantly increase the user's cognitive workload. These promising results suggest that acoustic touch can provide a wearable and effective method of sensory augmentation.

Overview

CoChat Summary

TL;DR

This study evaluated a wearable “acoustic touch” system (the Foveated Audio Device, FAD) that sonifies objects in view to help people who are blind find and reach items. In a 14-participant study, the FAD enabled object recognition and reaching without significantly increasing cognitive workload, though object-recognition robustness remained a major technical limitation.

Key findings

  • In the seated task, BLV participants’ overall success with the FAD was 81.66±13.72%, compared with 90.71±11.70% for Clock and 85±13.54% for Speakers; the paper describes FAD performance as comparable to the idealized conditions.
  • Blindfolded sighted participants performed worse with the FAD in the seated task: 73.01±18.92% vs 93.78±7.39% for Clock and 89.03±16.66% for Speakers; the FAD was significantly lower than both Clock and Speakers (Z = 2.21, p = 0.027, r = 0.83 for both comparisons).
  • In the standing task, all participants successfully found the target item, with an average of 1.67 reach attempts to find the target (BLV = 1.76±0.75; blindfolded sighted = 1.52±0.34).
  • The device did not significantly increase cognitive workload overall; the paper reports no change in most NASA-TLX categories, though BLV participants reported increased effort in the standing task and sighted participants reported higher physical demand as the study progressed.

Methods

  • The authors developed a wearable FAD using NReal Light AR glasses and an OPPO Find X3 Pro smartphone, with YOLOv5m object recognition, stereo depth estimation, and auditory icons for four household items: book, bottle, bowl, and cup.
  • They recruited 17 participants, excluded 3 BLV participants, and analyzed a final sample of 14: 7 BLV and 7 blindfolded sighted participants; BLV mean age was 51.43±14.89, sighted mean age was 42.43±13.56.
  • The study used a training stage, a seated task with 60 trials across 3 conditions (20 FAD, 20 Clock, 20 Speakers), and a standing task with 3 trials; analyses included Wilcoxon rank sum and Wilcoxon signed-rank tests, plus NASA-TLX and physiological measures.

Limitations

  • The paper explicitly states two technical limitations: object recognition robustness was insufficient for real-world use, and the system caused significant heating and power consumption issues on the smartphone.
  • The authors note that the study limited recognition to four easily recognized object classes, which they say was necessary to ensure reliable performance during the user study; this narrows generalizability to real-world settings.
  • The text also makes evident that the standing-task motion-capture data were not collected because of tracking-area size limitations, and some analyses were based on relatively few unsuccessful trials, which the paper says reduced reliability for those comparisons.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0290431

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 (A) The study set-up for the seated task of the study. The participants were asked to use the information provided (either using the provided wearable device or an external source) to identify and memorize the item locations on the table. The participant would then be asked to search and reach for a specific item on the table using the memorized map (without any aids). (B) The standing task requires the participant to use the provided wearable device to search and reach for a target item situated among multiple distractor items (bottles).
Fig 1 (A) The study set-up for the seated task of the study. The participants were asked to use the information provided (either using the provided wearable device or an external source) to identify and memorize the item locations on the table. The participant would then be asked to search and reach for a specific item on the table using the memorized map (without any aids). (B) The standing task requires the participant to use the provided wearable device to search and reach for a target item situated among multiple distractor items (bottles).CC BY · PubMed Central
Fig 2 (A) The NReal AR glasses worn by a participant. The grey spheres are the reflective markers that tracked the participant’s head movement through motion capture. The participant’s face has been blurred for privacy. (B) The household items used during the study. The items (from left to right) are the bowl, book, cup, and bottle. (C) The Sony XB13 Bluetooth speakers and the user response button were used during the study.
Fig 2 (A) The NReal AR glasses worn by a participant. The grey spheres are the reflective markers that tracked the participant’s head movement through motion capture. The participant’s face has been blurred for privacy. (B) The household items used during the study. The items (from left to right) are the bowl, book, cup, and bottle. (C) The Sony XB13 Bluetooth speakers and the user response button were used during the study.CC BY · PubMed Central
Fig 3 A demonstration of the NReal glasses, OPPO android smartphone, and the sonification app being used to detect a cup and bottle within the user’s FOV.
Fig 3 A demonstration of the NReal glasses, OPPO android smartphone, and the sonification app being used to detect a cup and bottle within the user’s FOV.CC BY · PubMed Central
Fig 4 A block diagram outlining the connections between the technical components and the study’s users (researchers and participants). The arrows represent the flow of information with the text specifying the type of information.
Fig 4 A block diagram outlining the connections between the technical components and the study’s users (researchers and participants). The arrows represent the flow of information with the text specifying the type of information.CC BY · PubMed Central
Fig 5 An outline of the single trial protocol for the seated task. The red circle indicates the non-automated aspect where the researcher or participant chooses to advance the trial through a key or button press.
Fig 5 An outline of the single trial protocol for the seated task. The red circle indicates the non-automated aspect where the researcher or participant chooses to advance the trial through a key or button press.CC BY · PubMed Central
Fig 6 (A) depicts the hand, head, and item tracking through motion capture in this study. (B) Plots of the hand trajectory (z-x plane, overhead view) for every FAD trial by the participants. The blue dots represent the hand’s starting position with an offset always initialized at (0,0). The red dots represent reached item position. Position 1–5 refers to reaching positions from the left of the user to the right.
Fig 6 (A) depicts the hand, head, and item tracking through motion capture in this study. (B) Plots of the hand trajectory (z-x plane, overhead view) for every FAD trial by the participants. The blue dots represent the hand’s starting position with an offset always initialized at (0,0). The red dots represent reached item position. Position 1–5 refers to reaching positions from the left of the user to the right.CC BY · PubMed Central
Fig 7 (A) The average with standard error bars of the seated task success rate sorted by each condition for both the BLV and blindfolded sighted participants. (B) The average with standard error bars of each item’s detection rate when using the FAD to search for the items on the table.
Fig 7 (A) The average with standard error bars of the seated task success rate sorted by each condition for both the BLV and blindfolded sighted participants. (B) The average with standard error bars of each item’s detection rate when using the FAD to search for the items on the table.CC BY · PubMed Central
Fig 8 (A) The average with standard error bars of the seated task success rate sorted by each item and condition for both the BLV and blindfolded sighted participants. (B) The average with standard error bars of the number of attempts to reach for the target item during the standing task.
Fig 8 (A) The average with standard error bars of the seated task success rate sorted by each item and condition for both the BLV and blindfolded sighted participants. (B) The average with standard error bars of the number of attempts to reach for the target item during the standing task.CC BY · PubMed Central
Fig 9 The box plots show the hand reaching speed, trajectory-to-optimal ratio, and the reaching time for the BLV and blindfolded sighted participant groups.
Fig 9 The box plots show the hand reaching speed, trajectory-to-optimal ratio, and the reaching time for the BLV and blindfolded sighted participant groups.CC BY · PubMed Central
Fig 10 The box plots for the total head rotation and head rotational speed during the scanning-for-items stage of the FAD condition are shown for the seated task. It should be noted that the item could be placed directly on either side of the participant; a single head sweep to search the table would be 180° rotation.
Fig 10 The box plots for the total head rotation and head rotational speed during the scanning-for-items stage of the FAD condition are shown for the seated task. It should be noted that the item could be placed directly on either side of the participant; a single head sweep to search the table would be 180° rotation.CC BY · PubMed Central
Fig 11 The average values with standard error bars are shown for each NASA-TLX category and the total (RAW-TLX) workload value for the BLV and Sighted participant groups. The scoring categories are Mental, Physical (Phys), Temporal (Temp), Performance (Per), Effort (Eff), Frustration (Frus), and Total Raw-TLX score. B.S. refers to the Blindfolded Sighted participants.
Fig 11 The average values with standard error bars are shown for each NASA-TLX category and the total (RAW-TLX) workload value for the BLV and Sighted participant groups. The scoring categories are Mental, Physical (Phys), Temporal (Temp), Performance (Per), Effort (Eff), Frustration (Frus), and Total Raw-TLX score. B.S. refers to the Blindfolded Sighted participants.CC BY · PubMed Central

Verification

Claims the paper makes

4 Valid · 0 Not valid · 13 Not checked · 107 Own results and interpretations

70 more claims

Create a free account to see all 124 claims and to verify the 4 references still open.

See all claims — free account
  • Probably not a substantial claim. 32
  • It may state two separate claims as one. 26

Create a free account to see these claims.

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentz = 2.21reported p = .027 · recomputed p = .027

    “(Z = 2.21, p = .027”

  • Consistentz = 2.2reported p = .028 · recomputed p = .028

    “(Z = 2.20, p = .028”

  • Consistentz = 2.03reported p = .043 · recomputed p = .042

    “(Z = 2.03, p = .043”

  • Consistentz = 2.2reported p = .028 · recomputed p = .028

    “Z = 2.20, p = .028”

  • Consistentz = 1.99reported p = .046 · recomputed p = .047

    “Z = 1.99, p = .046”

5 of 10 · the rest aren’t loaded

5 more recomputed tests

Create a free account to see every test the Stats check recomputed.

See all tests — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Citations

Cites

Show all 58 references

Research works printed but not linked: 1

  1. Dingler T, Lindsay J, Walker BN, et al. Learnability of sound cues for environmental features: Auditory icons, earcons, spearcons, and speech. In: Proceedings of the 14th International Conference on Auditory Display, Paris, France; 2008. p. 1–6. Available from: http://www.icad.org/Proceedings/2008/DinglerLindsay2008.pdf.

Cited by

See all 16

Metadata

Authors
Zhu, H. Y., Hossain, S. N., Jin, C., Singh, A. K., Nguyen, M. T. D., Deverell, L., Nguyen, V., Gates, F. S., Fernandez, I. G., Melencio, M. V., Bell, J.-A. R., & Lin, C.-T.
Affiliations
  • University of Technology Sydney (AU)
  • The University of Sydney (AU)
Corresponding author
Zhu, Howe Yuan
Venue
PLoS ONE
Published
2023-10-25
Volume
18
Issue
10
Pages
e0290431
ISSN
1932-6203 (ISSN-L)
OpenAlex
W4387950336
Cited by
16
References
57
Citations by year
2026: 4 · 2025: 6 · 2024: 6
FWCI
2.39
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Australian Cooperative Research Centres Projects (CRCPXI000007)
  • the ARIA research
  • the University of Technology Sydney
  • the University of Sydney
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
  • Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
Keywords
assistive technology for the blind, low vision, spatial audio, sonification, auditory icons, object recognition, sensory augmentation, mental workload
MeSH terms
Smart Glasses, Acoustics, Blindness, Humans, Speech, Vision, Ocular, Touch Perception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: An investigation into the effectiveness of using acoustic touch to assist people who are blind. https://app.cochat.ai/papers/10.1371/journal.pone.0290431
The paper
Zhu, H. Y., Hossain, S. N., Jin, C., Singh, A. K., Nguyen, M. T. D., Deverell, L., Nguyen, V., Gates, F. S., Fernandez, I. G., Melencio, M. V., Bell, J.-A. R., & Lin, C.-T. (2023). An investigation into the effectiveness of using acoustic touch to assist people who are blind. PLoS ONE. https://doi.org/10.1371/journal.pone.0290431

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.