An investigation into the effectiveness of using acoustic touch to assist people who are blind
Open accessThe DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index; 5 of the 37 sources its claims cite were checked for support.
Abstract
Wearable smart glasses are an emerging technology gaining popularity in the assistive technologies industry. Smart glasses aids typically leverage computer vision and other sensory information to translate the wearer's surrounding into computer-synthesized speech. In this work, we explored the potential of a new technique known as "acoustic touch" to provide a wearable spatial audio solution for assisting people who are blind in finding objects. In contrast to traditional systems, this technique uses smart glasses to sonify objects into distinct sound auditory icons when the object enters the device's field of view. We developed a wearable Foveated Audio Device to study the efficacy and usability of using acoustic touch to search, memorize, and reach items. Our evaluation study involved 14 participants, 7 blind or low-visioned and 7 blindfolded sighted (as a control group) participants. We compared the wearable device to two idealized conditions, a verbal clock face description and a sequential audio presentation through external speakers. We found that the wearable device can effectively aid the recognition and reaching of an object. We also observed that the device does not significantly increase the user's cognitive workload. These promising results suggest that acoustic touch can provide a wearable and effective method of sensory augmentation.
Overview
CoChat Summary
TL;DR
This study evaluated a wearable “acoustic touch” system (the Foveated Audio Device, FAD) that sonifies objects in view to help people who are blind find and reach items. In a 14-participant study, the FAD enabled object recognition and reaching without significantly increasing cognitive workload, though object-recognition robustness remained a major technical limitation.
Key findings
- In the seated task, BLV participants’ overall success with the FAD was 81.66±13.72%, compared with 90.71±11.70% for Clock and 85±13.54% for Speakers; the paper describes FAD performance as comparable to the idealized conditions.
- Blindfolded sighted participants performed worse with the FAD in the seated task: 73.01±18.92% vs 93.78±7.39% for Clock and 89.03±16.66% for Speakers; the FAD was significantly lower than both Clock and Speakers (Z = 2.21, p = 0.027, r = 0.83 for both comparisons).
- In the standing task, all participants successfully found the target item, with an average of 1.67 reach attempts to find the target (BLV = 1.76±0.75; blindfolded sighted = 1.52±0.34).
- The device did not significantly increase cognitive workload overall; the paper reports no change in most NASA-TLX categories, though BLV participants reported increased effort in the standing task and sighted participants reported higher physical demand as the study progressed.
Methods
- The authors developed a wearable FAD using NReal Light AR glasses and an OPPO Find X3 Pro smartphone, with YOLOv5m object recognition, stereo depth estimation, and auditory icons for four household items: book, bottle, bowl, and cup.
- They recruited 17 participants, excluded 3 BLV participants, and analyzed a final sample of 14: 7 BLV and 7 blindfolded sighted participants; BLV mean age was 51.43±14.89, sighted mean age was 42.43±13.56.
- The study used a training stage, a seated task with 60 trials across 3 conditions (20 FAD, 20 Clock, 20 Speakers), and a standing task with 3 trials; analyses included Wilcoxon rank sum and Wilcoxon signed-rank tests, plus NASA-TLX and physiological measures.
Limitations
- The paper explicitly states two technical limitations: object recognition robustness was insufficient for real-world use, and the system caused significant heating and power consumption issues on the smartphone.
- The authors note that the study limited recognition to four easily recognized object classes, which they say was necessary to ensure reliable performance during the user study; this narrows generalizability to real-world settings.
- The text also makes evident that the standing-task motion-capture data were not collected because of tracking-area size limitations, and some analyses were based on relatively few unsuccessful trials, which the paper says reduced reliability for those comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
4 Valid · 0 Not valid · 13 Not checked · 107 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“All commercially available AT smart glasses, such as the OrCam [38], Eyedaptic [39], and Envision [31], use integrated front-facing cameras to perform object recognition resulting in some form of ‘acoustic touch’.”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 1 Introduction Assistive technology (AT) is an extensive research field that involves designing technologies to enable an individual with a sensory disability to overcome various physical, social, infrastructural and accessibility barriers in their activities of daily living (ADL) [1]. One central area within this field is developing assistive technology for people who are blind or have low vision (BLV). According to the World Health Organisation, it is estimated that blindness affects approximately 39 million people and, additionally, 246 million people have low vision [2]. Vision can be thought of as a primary and integrated sense, and it carries an important modality. The absence of vision affects many ADLs, and the lack of access to visual information impacts one’s ability to participate in a range of life and community activities [3]. Considerable research is dedicated to exploring the capability of AT [1]. With specialized Orientation and Mobility (O&M) training [4, 5] to assist people who are BLV or who have low vision. The common goal for AT and specialist training for people who are BLV is to achieve equitable access to facilitate independent living, individual career choices, and provide the ability to independently perform ADL, including navigation/travel [6], hazard detection/avoidance [7, 8], locating specific household items or personal belongings [9], reading/writing [10], and engaging in social interactions (including using social media) [11]. Human performance is a complex integration of sensory and motor systems, and exploring every ADL outlined is not feasible. Therefore, this paper primarily focuses on locating specific household items or personal belongings. Sensory augmentation is a broad AT research area, including using visual, haptic/tactile (sense of touch), and auditory feedback as a mode of augmentation; this paper focuses on using auditory sensory augmentation for people with BLV. In this case, visual sensory augmentation methods are irrelevant to our end-user group. Haptic or tactile sensory augmentation is another popular AT form for BLV users. Past works have successfully translated visual information into wearable (helmets, belts, gloves, and smart canes) haptic/tactile feedback [12–15]. The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16]. Directional sense can only be achieved through sensory-motor coupling (relation between motor action and sensory feedback) and requires a higher level of physical and cognitive effect to gauge precise/complex spatial information. In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19]. Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22]. Many devices, both technological and mechanical in design, exist to address the various ADL needs of people who are BLV via sensory substitution and augmentation. Sensory substitution refers to converting an impaired sensory modality into another perceivable form to the user [23]. Sensory augmentation furthers this concept by using technology to augment or adapt sensory feedback to enhance the fidelity of information presented to the user [24]. The long cane is a good example of a sensory augmentation device that provides tactile sensory feedback as a mobility aid for independent travel [1]. Other more technologically-complicated examples are screen readers (text to speech), Braille displays (print to braille feedback) [25], handheld sonar devices (visual to auditory and tactile feedback) [6], and smartphone applications (Be My Eyes [26] and Seeing AI [27], visual/text to auditory feedback). The recent improvements in augmented reality (AR), wearable camera/sensor technology, and deep learning-based computer vision have led to the emergence of smart glasses as versatile and multifunctional AT [28, 29]. These systems tend to fuse a camera, Inertial Measurement Unit (IMU), microphone, Global Positioning System (GPS) and depth sensing data to provide services such as rendering environment/object/text/person as computer-synthesized speech, voice recognition control (for internet search or booking services), navigation, and voice/video communication (social call or contacting a human guide) [30, 31]. These smart glasses have many important benefits, such as being hands-free, inconspicuous in appearance, and multifunctional. Many smart glasses systems [30, 31] leverages artificial technology and speech audio feedback to deliver a reliable and robust real-time solution to aiding real-world navigation [32]. Smart glasses technology is growing in popularity and slowly being adopted within the BLV community. Inspired by human echolocation training [4, 33], we explored the concept of “acoustic touch”, coined by Jin et al. [9], which involves the use of head scanning and the activation of auditory icons (instead of computer-synthesized speech [31]) as objects appear within a defined field of view to convey the identity and location of various items to the user. Acoustic touch builds on previous research and incorporates sensory-motor coupling via head scanning and auditory icon sonification to enhance the spatial perception of close items [9]. We propose that the technique of acoustic touch offers these three advantages:
What the sources say
- Supportedcites [38] M Howard, 2020, Harnessing the Power of OrCam, Information Technology and Libraries source
In 'Harnessing the Power of OrCam' (single-section article, no pagination in our copy), Howard describes the OrCam MyEye as a commercially available, purchasable ($3,500 retail, six units bought with the IMLS grant) wearable assistive device that 'can be to either the left or right temple on your glasses using a magnetic docking device,' is a 'portable, wearable visual system ... via the use of artificial computer intelligence and augmented reality,' and that 'automatically reads any printed text ... and can identify money and other business and household items,' giving immediate spoken output ('will repeat back any text immediately'; 'stunningly accurate feedback'). That makes the cited work a valid identification of OrCam as one of the camera-based, object/text-recognition, audio-output AT glasses the citing sentence lists; the article does not use the words 'front-facing camera' or 'acoustic touch,' and the sweeping 'all commercially available AT smart glasses' generalization is the citing authors' own claim rather than something this source is offered for. (LLM, conf=0.80)
- Not matchedcites [39] Eyedaptic. AMD glasses: Macular degeneration eyewear; 2021. Available from: https://eyedaptic.com/.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [31] Envision. Envision glasses; 2022. Available from: https://www.letsenvision.com/glasses.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19].”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 1 Introduction Assistive technology (AT) is an extensive research field that involves designing technologies to enable an individual with a sensory disability to overcome various physical, social, infrastructural and accessibility barriers in their activities of daily living (ADL) [1]. One central area within this field is developing assistive technology for people who are blind or have low vision (BLV). According to the World Health Organisation, it is estimated that blindness affects approximately 39 million people and, additionally, 246 million people have low vision [2]. Vision can be thought of as a primary and integrated sense, and it carries an important modality. The absence of vision affects many ADLs, and the lack of access to visual information impacts one’s ability to participate in a range of life and community activities [3]. Considerable research is dedicated to exploring the capability of AT [1]. With specialized Orientation and Mobility (O&M) training [4, 5] to assist people who are BLV or who have low vision. The common goal for AT and specialist training for people who are BLV is to achieve equitable access to facilitate independent living, individual career choices, and provide the ability to independently perform ADL, including navigation/travel [6], hazard detection/avoidance [7, 8], locating specific household items or personal belongings [9], reading/writing [10], and engaging in social interactions (including using social media) [11]. Human performance is a complex integration of sensory and motor systems, and exploring every ADL outlined is not feasible. Therefore, this paper primarily focuses on locating specific household items or personal belongings. Sensory augmentation is a broad AT research area, including using visual, haptic/tactile (sense of touch), and auditory feedback as a mode of augmentation; this paper focuses on using auditory sensory augmentation for people with BLV. In this case, visual sensory augmentation methods are irrelevant to our end-user group. Haptic or tactile sensory augmentation is another popular AT form for BLV users. Past works have successfully translated visual information into wearable (helmets, belts, gloves, and smart canes) haptic/tactile feedback [12–15]. The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16]. Directional sense can only be achieved through sensory-motor coupling (relation between motor action and sensory feedback) and requires a higher level of physical and cognitive effect to gauge precise/complex spatial information. In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19]. Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22]. Many devices, both technological and mechanical in design, exist to address the various ADL needs of people who are BLV via sensory substitution and augmentation. Sensory substitution refers to converting an impaired sensory modality into another perceivable form to the user [23]. Sensory augmentation furthers this concept by using technology to augment or adapt sensory feedback to enhance the fidelity of information presented to the user [24]. The long cane is a good example of a sensory augmentation device that provides tactile sensory feedback as a mobility aid for independent travel [1]. Other more technologically-complicated examples are screen readers (text to speech), Braille displays (print to braille feedback) [25], handheld sonar devices (visual to auditory and tactile feedback) [6], and smartphone applications (Be My Eyes [26] and Seeing AI [27], visual/text to auditory feedback). The recent improvements in augmented reality (AR), wearable camera/sensor technology, and deep learning-based computer vision have led to the emergence of smart glasses as versatile and multifunctional AT [28, 29]. These systems tend to fuse a camera, Inertial Measurement Unit (IMU), microphone, Global Positioning System (GPS) and depth sensing data to provide services such as rendering environment/object/text/person as computer-synthesized speech, voice recognition control (for internet search or booking services), navigation, and voice/video communication (social call or contacting a human guide) [30, 31]. These smart glasses have many important benefits, such as being hands-free, inconspicuous in appearance, and multifunctional. Many smart glasses systems [30, 31] leverages artificial technology and speech audio feedback to deliver a reliable and robust real-time solution to aiding real-world navigation [32]. Smart glasses technology is growing in popularity and slowly being adopted within the BLV community. Inspired by human echolocation training [4, 33], we explored the concept of “acoustic touch”, coined by Jin et al. [9], which involves the use of head scanning and the activation of auditory icons (instead of computer-synthesized speech [31]) as objects appear within a defined field of view to convey the identity and location of various items to the user. Acoustic touch builds on previous research and incorporates sensory-motor coupling via head scanning and auditory icon sonification to enhance the spatial perception of close items [9]. We propose that the technique of acoustic touch offers these three advantages:
What the sources say
- Not checked — no full text of the sourcecites [17] C Carello, 1998, Perception of Object Length by Sound, Psychological Science source
- Partly supportedcites [18] F Schumann, 2017, Sensory augmentation: integration of an auditory compass signal into human perception of space, Scientific Reports sourceExperimental — accuracy not yet measured
The Discussion (page not available) says the auditory compass can interface with existing spatial processes through natural auditory localization, and reports that most participants immediately perceived its sound as coming from a stable direction. Experiment 2 reports accurate use after 25 minutes of training. The paper does not establish the broader claim that human direction hearing generally requires less training to perceive spatial information accurately. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites [19] K Papadopoulos, 2017, Cognitive maps of individuals with blindness for familiar and unfamiliar spaces: Construction through audio-tactile maps and walked experience, Computers in Human Behavior source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16].”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 1 Introduction Assistive technology (AT) is an extensive research field that involves designing technologies to enable an individual with a sensory disability to overcome various physical, social, infrastructural and accessibility barriers in their activities of daily living (ADL) [1]. One central area within this field is developing assistive technology for people who are blind or have low vision (BLV). According to the World Health Organisation, it is estimated that blindness affects approximately 39 million people and, additionally, 246 million people have low vision [2]. Vision can be thought of as a primary and integrated sense, and it carries an important modality. The absence of vision affects many ADLs, and the lack of access to visual information impacts one’s ability to participate in a range of life and community activities [3]. Considerable research is dedicated to exploring the capability of AT [1]. With specialized Orientation and Mobility (O&M) training [4, 5] to assist people who are BLV or who have low vision. The common goal for AT and specialist training for people who are BLV is to achieve equitable access to facilitate independent living, individual career choices, and provide the ability to independently perform ADL, including navigation/travel [6], hazard detection/avoidance [7, 8], locating specific household items or personal belongings [9], reading/writing [10], and engaging in social interactions (including using social media) [11]. Human performance is a complex integration of sensory and motor systems, and exploring every ADL outlined is not feasible. Therefore, this paper primarily focuses on locating specific household items or personal belongings. Sensory augmentation is a broad AT research area, including using visual, haptic/tactile (sense of touch), and auditory feedback as a mode of augmentation; this paper focuses on using auditory sensory augmentation for people with BLV. In this case, visual sensory augmentation methods are irrelevant to our end-user group. Haptic or tactile sensory augmentation is another popular AT form for BLV users. Past works have successfully translated visual information into wearable (helmets, belts, gloves, and smart canes) haptic/tactile feedback [12–15]. The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16]. Directional sense can only be achieved through sensory-motor coupling (relation between motor action and sensory feedback) and requires a higher level of physical and cognitive effect to gauge precise/complex spatial information. In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19]. Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22]. Many devices, both technological and mechanical in design, exist to address the various ADL needs of people who are BLV via sensory substitution and augmentation. Sensory substitution refers to converting an impaired sensory modality into another perceivable form to the user [23]. Sensory augmentation furthers this concept by using technology to augment or adapt sensory feedback to enhance the fidelity of information presented to the user [24]. The long cane is a good example of a sensory augmentation device that provides tactile sensory feedback as a mobility aid for independent travel [1]. Other more technologically-complicated examples are screen readers (text to speech), Braille displays (print to braille feedback) [25], handheld sonar devices (visual to auditory and tactile feedback) [6], and smartphone applications (Be My Eyes [26] and Seeing AI [27], visual/text to auditory feedback). The recent improvements in augmented reality (AR), wearable camera/sensor technology, and deep learning-based computer vision have led to the emergence of smart glasses as versatile and multifunctional AT [28, 29]. These systems tend to fuse a camera, Inertial Measurement Unit (IMU), microphone, Global Positioning System (GPS) and depth sensing data to provide services such as rendering environment/object/text/person as computer-synthesized speech, voice recognition control (for internet search or booking services), navigation, and voice/video communication (social call or contacting a human guide) [30, 31]. These smart glasses have many important benefits, such as being hands-free, inconspicuous in appearance, and multifunctional. Many smart glasses systems [30, 31] leverages artificial technology and speech audio feedback to deliver a reliable and robust real-time solution to aiding real-world navigation [32]. Smart glasses technology is growing in popularity and slowly being adopted within the BLV community. Inspired by human echolocation training [4, 33], we explored the concept of “acoustic touch”, coined by Jin et al. [9], which involves the use of head scanning and the activation of auditory icons (instead of computer-synthesized speech [31]) as objects appear within a defined field of view to convey the identity and location of various items to the user. Acoustic touch builds on previous research and incorporates sensory-motor coupling via head scanning and auditory icon sonification to enhance the spatial perception of close items [9]. We propose that the technique of acoustic touch offers these three advantages:
What the sources say
- Partly supportedcites [16] SM Kärcher, 2012, Sensory Augmentation for the Blind, Frontiers in Human Neuroscience sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) says integrating tactile sensory information requires learning complex perceptual skills, and the Discussion (page unavailable) links training and prior cognitive grounding to spatial perception. The paper does not establish that significant training is the main challenge across haptic augmentation or that it produces semantic awareness of the surroundings; it notes that a subject with prior spatial grounding could benefit immediately. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22].”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 1 Introduction Assistive technology (AT) is an extensive research field that involves designing technologies to enable an individual with a sensory disability to overcome various physical, social, infrastructural and accessibility barriers in their activities of daily living (ADL) [1]. One central area within this field is developing assistive technology for people who are blind or have low vision (BLV). According to the World Health Organisation, it is estimated that blindness affects approximately 39 million people and, additionally, 246 million people have low vision [2]. Vision can be thought of as a primary and integrated sense, and it carries an important modality. The absence of vision affects many ADLs, and the lack of access to visual information impacts one’s ability to participate in a range of life and community activities [3]. Considerable research is dedicated to exploring the capability of AT [1]. With specialized Orientation and Mobility (O&M) training [4, 5] to assist people who are BLV or who have low vision. The common goal for AT and specialist training for people who are BLV is to achieve equitable access to facilitate independent living, individual career choices, and provide the ability to independently perform ADL, including navigation/travel [6], hazard detection/avoidance [7, 8], locating specific household items or personal belongings [9], reading/writing [10], and engaging in social interactions (including using social media) [11]. Human performance is a complex integration of sensory and motor systems, and exploring every ADL outlined is not feasible. Therefore, this paper primarily focuses on locating specific household items or personal belongings. Sensory augmentation is a broad AT research area, including using visual, haptic/tactile (sense of touch), and auditory feedback as a mode of augmentation; this paper focuses on using auditory sensory augmentation for people with BLV. In this case, visual sensory augmentation methods are irrelevant to our end-user group. Haptic or tactile sensory augmentation is another popular AT form for BLV users. Past works have successfully translated visual information into wearable (helmets, belts, gloves, and smart canes) haptic/tactile feedback [12–15]. The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16]. Directional sense can only be achieved through sensory-motor coupling (relation between motor action and sensory feedback) and requires a higher level of physical and cognitive effect to gauge precise/complex spatial information. In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19]. Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22]. Many devices, both technological and mechanical in design, exist to address the various ADL needs of people who are BLV via sensory substitution and augmentation. Sensory substitution refers to converting an impaired sensory modality into another perceivable form to the user [23]. Sensory augmentation furthers this concept by using technology to augment or adapt sensory feedback to enhance the fidelity of information presented to the user [24]. The long cane is a good example of a sensory augmentation device that provides tactile sensory feedback as a mobility aid for independent travel [1]. Other more technologically-complicated examples are screen readers (text to speech), Braille displays (print to braille feedback) [25], handheld sonar devices (visual to auditory and tactile feedback) [6], and smartphone applications (Be My Eyes [26] and Seeing AI [27], visual/text to auditory feedback). The recent improvements in augmented reality (AR), wearable camera/sensor technology, and deep learning-based computer vision have led to the emergence of smart glasses as versatile and multifunctional AT [28, 29]. These systems tend to fuse a camera, Inertial Measurement Unit (IMU), microphone, Global Positioning System (GPS) and depth sensing data to provide services such as rendering environment/object/text/person as computer-synthesized speech, voice recognition control (for internet search or booking services), navigation, and voice/video communication (social call or contacting a human guide) [30, 31]. These smart glasses have many important benefits, such as being hands-free, inconspicuous in appearance, and multifunctional. Many smart glasses systems [30, 31] leverages artificial technology and speech audio feedback to deliver a reliable and robust real-time solution to aiding real-world navigation [32]. Smart glasses technology is growing in popularity and slowly being adopted within the BLV community. Inspired by human echolocation training [4, 33], we explored the concept of “acoustic touch”, coined by Jin et al. [9], which involves the use of head scanning and the activation of auditory icons (instead of computer-synthesized speech [31]) as objects appear within a defined field of view to convey the identity and location of various items to the user. Acoustic touch builds on previous research and incorporates sensory-motor coupling via head scanning and auditory icon sonification to enhance the spatial perception of close items [9]. We propose that the technique of acoustic touch offers these three advantages:
What the sources say
- Not checked — no full text of the sourcecites [15] O Lahav, 2008, Construction of cognitive maps of unknown spaces using a multi-sensory virtual environment for people who are blind, Computers in Human Behavior source
- Not checked — no full text of the sourcecites [19] K Papadopoulos, 2017, Cognitive maps of individuals with blindness for familiar and unfamiliar spaces: Construction through audio-tactile maps and walked experience, Computers in Human Behavior source
- Partly supportedcites [20] S Abboud, 2014, EyeMusic: Introducing a “visual” colorful experience for the blind using auditory sensory substitution, Restorative Neurology and Neuroscience sourceExperimental — accuracy not yet measured
The Introduction reports that users of sensory-substitution devices can localize stimuli, navigate, and avoid obstacles. But the Behavioral experiment tested identification of simple shapes and colors after 2–3 hours of training, not spatial awareness of the surrounding environment; the Discussion says locating real objects was observed only anecdotally after more training. The source therefore supports the spatial-information component, but not the claim that it was achieved with very little training. (LLM, conf=0.88)
- Partly supportedcites [21] A Neugebauer, 2020, Navigation aid for blind persons by visual-to-auditory sensory substitution: A pilot study, PLOS ONE sourceExperimental — accuracy not yet measured
The Conclusion reports that blind participants navigated with the auditory device immediately after instructions and recognized 3D objects after limited use. But the tasks used a simulated environment synchronized with real-world objects; the Discussion says the device could not yet capture unmarked real-world obstacles. The source therefore supports success with little training in the test setting, not accurate information about a user's surroundings generally. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites [22] CHS Chang, 2022, Design and evaluation of a multi-sensory scaffolding gamification science course with mobile technology for learners with total blindness, Computers in Human Behavior source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The blindfolded sighted participants were significantly lower in success rate in the FAD condition compared to the Clock (Z = 2.21, p = 0.027, and r = 0.83) and Speaker condition (Z = 2.21, p = 0.027, and r = 0.83).”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 3 Results > 3.1 Seated task performance
plainn = 5 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
70 more claims
Create a free account to see all 124 claims and to verify the 4 references still open.
- Probably not a substantial claim. 32
- It may state two separate claims as one. 26
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = 2.21reported p = .027 · recomputed p = .027
“(Z = 2.21, p = .027”
- Consistentz = 2.2reported p = .028 · recomputed p = .028
“(Z = 2.20, p = .028”
- Consistentz = 2.03reported p = .043 · recomputed p = .042
“(Z = 2.03, p = .043”
- Consistentz = 2.2reported p = .028 · recomputed p = .028
“Z = 2.20, p = .028”
- Consistentz = 1.99reported p = .046 · recomputed p = .047
“Z = 1.99, p = .046”
5 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Lecture Notes in Computer Science 1205
(1999) · Industrial Robot: An International Journal · doi:10.1108/ir.1999.04926fae.001 · Cited by 38,662
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297
Sensory substitution and the human-machine interface
Bach-y-Rita, P., & W. Kercel, S. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.10.013 · Cited by 858
Visual Prosody and Speech Intelligibility: Head Movement Improves Auditory Speech Perception
Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., & Vatikiotis-Bateson, E. (2004) · Psychological Science · doi:10.1111/j.0963-7214.2004.01502010.x · Cited by 482
Bhowmick, A., & Hazarika, S. M. (2017) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-016-0235-6 · Cited by 271
A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching
Gehrig, S. K., Eberli, F., & Meyer, T. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-04667-4_14 · Cited by 270
A summary of research investigating echolocation abilities of blind and sighted humans
Kolarik, A. J., Cirstea, S., Pardhan, S., & Moore, B. C. J. (2014) · Hearing Research · doi:10.1016/j.heares.2014.01.010 · Cited by 270
Static and Dynamic Hand Gesture Recognition in Depth Data Using Dynamic Time Warping
Plouffe, G., & Cretu, A.-M. (2016) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2015.2498560 · Cited by 266
Haptic wearables as sensory replacement, sensory augmentation and trainer – a review
Shull, P. B., & Damian, D. D. (2015) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-015-0055-z · Cited by 245
Research works printed but not linked: 1
- Dingler T, Lindsay J, Walker BN, et al. Learnability of sound cues for environmental features: Auditory icons, earcons, spearcons, and speech. In: Proceedings of the 14th International Conference on Auditory Display, Paris, France; 2008. p. 1–6. Available from: http://www.icad.org/Proceedings/2008/DinglerLindsay2008.pdf.
Cited by
A Human-Centric Metaverse Enabled by Brain-Computer Interface: A Survey
Zhu, H. Y., Hieu, N. Q., Hoang, D. T., Nguyen, D. N., & Lin, C. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3387124 · Cited by 100
“BCI can also be integrated with other assistive technologies [25] or input modalities [26] to create multimodal interfaces [27].”
BackgroundSelf-citationLimweshasin, N., Castro, I. A. A., Korposh, S., Morgan, S. P., Hayes‐Gill, B., Faghy, M. A., & Correia, R. G. (2024) · Sensors · doi:10.3390/s24237476 · Cited by 10
Nguyen, M. T. D., Zhu, H. Y., Sun, H., et al. (2024) · doi:10.1145/3688862.3689115 · Cited by 6
“The human brain utilises binaural hearing to localise sounds both passively and actively [9, 19, 23].”
Self-citationMinton, A. G., Zhu, H. Y., Chen, H., Wang, Y., Zhuang, Z., Notaro, G. M., Galvan-Garza, R. C., Allen, J. P., Ziegler, M. D., & Lin, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713402 · Cited by 5
Self-citationAtes, Ö., Pandey, G., Gousiopoulos, A., & Soldatos, T. G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1424371 · Cited by 3
Metadata
- Authors
- Zhu, H. Y., Hossain, S. N., Jin, C., Singh, A. K., Nguyen, M. T. D., Deverell, L., Nguyen, V., Gates, F. S., Fernandez, I. G., Melencio, M. V., Bell, J.-A. R., & Lin, C.-T.
- Affiliations
- University of Technology Sydney (AU)
- The University of Sydney (AU)
- Corresponding author
- Zhu, Howe Yuan
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-10-25
- Volume
- 18
- Issue
- 10
- Pages
- e0290431
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC10599575
- OpenAlex
- W4387950336
- Semantic Scholar
- 2b33b37418485873173142769007a23aa1a99db1
- Cited by
- 16
- References
- 57
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 6
- FWCI
- 2.39
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Australian Cooperative Research Centres Projects (CRCPXI000007)
- the ARIA research
- the University of Technology Sydney
- the University of Sydney
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Keywords
- assistive technology for the blind, low vision, spatial audio, sonification, auditory icons, object recognition, sensory augmentation, mental workload
- MeSH terms
- Smart Glasses, Acoustics, Blindness, Humans, Speech, Vision, Ocular, Touch Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An investigation into the effectiveness of using acoustic touch to assist people who are blind. https://app.cochat.ai/papers/10.1371/journal.pone.0290431
- The paper
- Zhu, H. Y., Hossain, S. N., Jin, C., Singh, A. K., Nguyen, M. T. D., Deverell, L., Nguyen, V., Gates, F. S., Fernandez, I. G., Melencio, M. V., Bell, J.-A. R., & Lin, C.-T. (2023). An investigation into the effectiveness of using acoustic touch to assist people who are blind. PLoS ONE. https://doi.org/10.1371/journal.pone.0290431
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.