An investigation into the effectiveness of using acoustic touch to assist people who are blind
Open accessThe DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index; 5 of the 37 sources its claims cite were checked for support.
Abstract
Wearable smart glasses are an emerging technology gaining popularity in the assistive technologies industry. Smart glasses aids typically leverage computer vision and other sensory information to translate the wearer's surrounding into computer-synthesized speech. In this work, we explored the potential of a new technique known as "acoustic touch" to provide a wearable spatial audio solution for assisting people who are blind in finding objects. In contrast to traditional systems, this technique uses smart glasses to sonify objects into distinct sound auditory icons when the object enters the device's field of view. We developed a wearable Foveated Audio Device to study the efficacy and usability of using acoustic touch to search, memorize, and reach items. Our evaluation study involved 14 participants, 7 blind or low-visioned and 7 blindfolded sighted (as a control group) participants. We compared the wearable device to two idealized conditions, a verbal clock face description and a sequential audio presentation through external speakers. We found that the wearable device can effectively aid the recognition and reaching of an object. We also observed that the device does not significantly increase the user's cognitive workload. These promising results suggest that acoustic touch can provide a wearable and effective method of sensory augmentation.
Overview
CoChat Summary
TL;DR
This study evaluated a wearable “acoustic touch” system (the Foveated Audio Device, FAD) that sonifies objects in view to help people who are blind find and reach items. In a 14-participant study, the FAD enabled object recognition and reaching without significantly increasing cognitive workload, though object-recognition robustness remained a major technical limitation.
Key findings
- In the seated task, BLV participants’ overall success with the FAD was 81.66±13.72%, compared with 90.71±11.70% for Clock and 85±13.54% for Speakers; the paper describes FAD performance as comparable to the idealized conditions.
- Blindfolded sighted participants performed worse with the FAD in the seated task: 73.01±18.92% vs 93.78±7.39% for Clock and 89.03±16.66% for Speakers; the FAD was significantly lower than both Clock and Speakers (Z = 2.21, p = 0.027, r = 0.83 for both comparisons).
- In the standing task, all participants successfully found the target item, with an average of 1.67 reach attempts to find the target (BLV = 1.76±0.75; blindfolded sighted = 1.52±0.34).
- The device did not significantly increase cognitive workload overall; the paper reports no change in most NASA-TLX categories, though BLV participants reported increased effort in the standing task and sighted participants reported higher physical demand as the study progressed.
Methods
- The authors developed a wearable FAD using NReal Light AR glasses and an OPPO Find X3 Pro smartphone, with YOLOv5m object recognition, stereo depth estimation, and auditory icons for four household items: book, bottle, bowl, and cup.
- They recruited 17 participants, excluded 3 BLV participants, and analyzed a final sample of 14: 7 BLV and 7 blindfolded sighted participants; BLV mean age was 51.43±14.89, sighted mean age was 42.43±13.56.
- The study used a training stage, a seated task with 60 trials across 3 conditions (20 FAD, 20 Clock, 20 Speakers), and a standing task with 3 trials; analyses included Wilcoxon rank sum and Wilcoxon signed-rank tests, plus NASA-TLX and physiological measures.
Limitations
- The paper explicitly states two technical limitations: object recognition robustness was insufficient for real-world use, and the system caused significant heating and power consumption issues on the smartphone.
- The authors note that the study limited recognition to four easily recognized object classes, which they say was necessary to ensure reliable performance during the user study; this narrows generalizability to real-world settings.
- The text also makes evident that the standing-task motion-capture data were not collected because of tracking-area size limitations, and some analyses were based on relatively few unsuccessful trials, which the paper says reduced reliability for those comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
4 Valid · 0 Not valid · 13 Not checked · 107 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“All commercially available AT smart glasses, such as the OrCam [38], Eyedaptic [39], and Envision [31], use integrated front-facing cameras to perform object recognition resulting in some form of ‘acoustic touch’.”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 1 Introduction Assistive technology (AT) is an extensive research field that involves designing technologies to enable an individual with a sensory disability to overcome various physical, social, infrastructural and accessibility barriers in their activities of daily living (ADL) [1]. One central area within this field is developing assistive technology for people who are blind or have low vision (BLV). According to the World Health Organisation, it is estimated that blindness affects approximately 39 million people and, additionally, 246 million people have low vision [2]. Vision can be thought of as a primary and integrated sense, and it carries an important modality. The absence of vision affects many ADLs, and the lack of access to visual information impacts one’s ability to participate in a range of life and community activities [3]. Considerable research is dedicated to exploring the capability of AT [1]. With specialized Orientation and Mobility (O&M) training [4, 5] to assist people who are BLV or who have low vision. The common goal for AT and specialist training for people who are BLV is to achieve equitable access to facilitate independent living, individual career choices, and provide the ability to independently perform ADL, including navigation/travel [6], hazard detection/avoidance [7, 8], locating specific household items or personal belongings [9], reading/writing [10], and engaging in social interactions (including using social media) [11]. Human performance is a complex integration of sensory and motor systems, and exploring every ADL outlined is not feasible. Therefore, this paper primarily focuses on locating specific household items or personal belongings. Sensory augmentation is a broad AT research area, including using visual, haptic/tactile (sense of touch), and auditory feedback as a mode of augmentation; this paper focuses on using auditory sensory augmentation for people with BLV. In this case, visual sensory augmentation methods are irrelevant to our end-user group. Haptic or tactile sensory augmentation is another popular AT form for BLV users. Past works have successfully translated visual information into wearable (helmets, belts, gloves, and smart canes) haptic/tactile feedback [12–15]. The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16]. Directional sense can only be achieved through sensory-motor coupling (relation between motor action and sensory feedback) and requires a higher level of physical and cognitive effect to gauge precise/complex spatial information. In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19]. Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22]. Many devices, both technological and mechanical in design, exist to address the various ADL needs of people who are BLV via sensory substitution and augmentation. Sensory substitution refers to converting an impaired sensory modality into another perceivable form to the user [23]. Sensory augmentation furthers this concept by using technology to augment or adapt sensory feedback to enhance the fidelity of information presented to the user [24]. The long cane is a good example of a sensory augmentation device that provides tactile sensory feedback as a mobility aid for independent travel [1]. Other more technologically-complicated examples are screen readers (text to speech), Braille displays (print to braille feedback) [25], handheld sonar devices (visual to auditory and tactile feedback) [6], and smartphone applications (Be My Eyes [26] and Seeing AI [27], visual/text to auditory feedback). The recent improvements in augmented reality (AR), wearable camera/sensor technology, and deep learning-based computer vision have led to the emergence of smart glasses as versatile and multifunctional AT [28, 29]. These systems tend to fuse a camera, Inertial Measurement Unit (IMU), microphone, Global Positioning System (GPS) and depth sensing data to provide services such as rendering environment/object/text/person as computer-synthesized speech, voice recognition control (for internet search or booking services), navigation, and voice/video communication (social call or contacting a human guide) [30, 31]. These smart glasses have many important benefits, such as being hands-free, inconspicuous in appearance, and multifunctional. Many smart glasses systems [30, 31] leverages artificial technology and speech audio feedback to deliver a reliable and robust real-time solution to aiding real-world navigation [32]. Smart glasses technology is growing in popularity and slowly being adopted within the BLV community. Inspired by human echolocation training [4, 33], we explored the concept of “acoustic touch”, coined by Jin et al. [9], which involves the use of head scanning and the activation of auditory icons (instead of computer-synthesized speech [31]) as objects appear within a defined field of view to convey the identity and location of various items to the user. Acoustic touch builds on previous research and incorporates sensory-motor coupling via head scanning and auditory icon sonification to enhance the spatial perception of close items [9]. We propose that the technique of acoustic touch offers these three advantages:
What the sources say
- Supportedcites [38] Howard M. Harnessing the Power of OrCam. Information Technology and Libraries. 2020; 39: 3. doi: 10.6017/ital.v39i3.12637 source
In 'Harnessing the Power of OrCam' (single-section article, no pagination in our copy), Howard describes the OrCam MyEye as a commercially available, purchasable ($3,500 retail, six units bought with the IMLS grant) wearable assistive device that 'can be to either the left or right temple on your glasses using a magnetic docking device,' is a 'portable, wearable visual system ... via the use of artificial computer intelligence and augmented reality,' and that 'automatically reads any printed text ... and can identify money and other business and household items,' giving immediate spoken output ('will repeat back any text immediately'; 'stunningly accurate feedback'). That makes the cited work a valid identification of OrCam as one of the camera-based, object/text-recognition, audio-output AT glasses the citing sentence lists; the article does not use the words 'front-facing camera' or 'acoustic touch,' and the sweeping 'all commercially available AT smart glasses' generalization is the citing authors' own claim rather than something this source is offered for. (LLM, conf=0.80)
- Not matchedcites [39] Eyedaptic. AMD glasses: Macular degeneration eyewear; 2021. Available from: https://eyedaptic.com/.
We searched by its title and found no matching paper.
- Not matchedcites [31] Envision. Envision glasses; 2022. Available from: https://www.letsenvision.com/glasses.
We searched by its title and found no matching paper.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19].”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 1 Introduction Assistive technology (AT) is an extensive research field that involves designing technologies to enable an individual with a sensory disability to overcome various physical, social, infrastructural and accessibility barriers in their activities of daily living (ADL) [1]. One central area within this field is developing assistive technology for people who are blind or have low vision (BLV). According to the World Health Organisation, it is estimated that blindness affects approximately 39 million people and, additionally, 246 million people have low vision [2]. Vision can be thought of as a primary and integrated sense, and it carries an important modality. The absence of vision affects many ADLs, and the lack of access to visual information impacts one’s ability to participate in a range of life and community activities [3]. Considerable research is dedicated to exploring the capability of AT [1]. With specialized Orientation and Mobility (O&M) training [4, 5] to assist people who are BLV or who have low vision. The common goal for AT and specialist training for people who are BLV is to achieve equitable access to facilitate independent living, individual career choices, and provide the ability to independently perform ADL, including navigation/travel [6], hazard detection/avoidance [7, 8], locating specific household items or personal belongings [9], reading/writing [10], and engaging in social interactions (including using social media) [11]. Human performance is a complex integration of sensory and motor systems, and exploring every ADL outlined is not feasible. Therefore, this paper primarily focuses on locating specific household items or personal belongings. Sensory augmentation is a broad AT research area, including using visual, haptic/tactile (sense of touch), and auditory feedback as a mode of augmentation; this paper focuses on using auditory sensory augmentation for people with BLV. In this case, visual sensory augmentation methods are irrelevant to our end-user group. Haptic or tactile sensory augmentation is another popular AT form for BLV users. Past works have successfully translated visual information into wearable (helmets, belts, gloves, and smart canes) haptic/tactile feedback [12–15]. The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16]. Directional sense can only be achieved through sensory-motor coupling (relation between motor action and sensory feedback) and requires a higher level of physical and cognitive effect to gauge precise/complex spatial information. In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19]. Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22]. Many devices, both technological and mechanical in design, exist to address the various ADL needs of people who are BLV via sensory substitution and augmentation. Sensory substitution refers to converting an impaired sensory modality into another perceivable form to the user [23]. Sensory augmentation furthers this concept by using technology to augment or adapt sensory feedback to enhance the fidelity of information presented to the user [24]. The long cane is a good example of a sensory augmentation device that provides tactile sensory feedback as a mobility aid for independent travel [1]. Other more technologically-complicated examples are screen readers (text to speech), Braille displays (print to braille feedback) [25], handheld sonar devices (visual to auditory and tactile feedback) [6], and smartphone applications (Be My Eyes [26] and Seeing AI [27], visual/text to auditory feedback). The recent improvements in augmented reality (AR), wearable camera/sensor technology, and deep learning-based computer vision have led to the emergence of smart glasses as versatile and multifunctional AT [28, 29]. These systems tend to fuse a camera, Inertial Measurement Unit (IMU), microphone, Global Positioning System (GPS) and depth sensing data to provide services such as rendering environment/object/text/person as computer-synthesized speech, voice recognition control (for internet search or booking services), navigation, and voice/video communication (social call or contacting a human guide) [30, 31]. These smart glasses have many important benefits, such as being hands-free, inconspicuous in appearance, and multifunctional. Many smart glasses systems [30, 31] leverages artificial technology and speech audio feedback to deliver a reliable and robust real-time solution to aiding real-world navigation [32]. Smart glasses technology is growing in popularity and slowly being adopted within the BLV community. Inspired by human echolocation training [4, 33], we explored the concept of “acoustic touch”, coined by Jin et al. [9], which involves the use of head scanning and the activation of auditory icons (instead of computer-synthesized speech [31]) as objects appear within a defined field of view to convey the identity and location of various items to the user. Acoustic touch builds on previous research and incorporates sensory-motor coupling via head scanning and auditory icon sonification to enhance the spatial perception of close items [9]. We propose that the technique of acoustic touch offers these three advantages:
What the sources say
- Not checked — no full text of the sourcecites [17] Carello C, Anderson KL, Kunkler-Peck AJ. Perception of Object Length by Sound. Psychological Science. 1998; 9(3): 211– 214. doi: 10.1111/1467-9280.00040 source
- Partly supportedcites [18] Schumann F, O’Regan JK. Sensory augmentation: integration of an auditory compass signal into human perception of space. Scientific Reports. 2017; 7(1): 42197. doi: 10.1038/srep42197 28195187 PMC5307328 sourceExperimental — accuracy not yet measured
The Discussion (page not available) says the auditory compass can interface with existing spatial processes through natural auditory localization, and reports that most participants immediately perceived its sound as coming from a stable direction. Experiment 2 reports accurate use after 25 minutes of training. The paper does not establish the broader claim that human direction hearing generally requires less training to perceive spatial information accurately. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites [19] Papadopoulos K, Koustriava E, Barouti M. Cognitive maps of individuals with blindness for familiar and unfamiliar spaces: Construction through audio-tactile maps and walked experience. Computers in Human Behavior. 2017; 75: 376– 384. doi: 10.1016/j.chb.2017.04.057 source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16].”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 1 Introduction Assistive technology (AT) is an extensive research field that involves designing technologies to enable an individual with a sensory disability to overcome various physical, social, infrastructural and accessibility barriers in their activities of daily living (ADL) [1]. One central area within this field is developing assistive technology for people who are blind or have low vision (BLV). According to the World Health Organisation, it is estimated that blindness affects approximately 39 million people and, additionally, 246 million people have low vision [2]. Vision can be thought of as a primary and integrated sense, and it carries an important modality. The absence of vision affects many ADLs, and the lack of access to visual information impacts one’s ability to participate in a range of life and community activities [3]. Considerable research is dedicated to exploring the capability of AT [1]. With specialized Orientation and Mobility (O&M) training [4, 5] to assist people who are BLV or who have low vision. The common goal for AT and specialist training for people who are BLV is to achieve equitable access to facilitate independent living, individual career choices, and provide the ability to independently perform ADL, including navigation/travel [6], hazard detection/avoidance [7, 8], locating specific household items or personal belongings [9], reading/writing [10], and engaging in social interactions (including using social media) [11]. Human performance is a complex integration of sensory and motor systems, and exploring every ADL outlined is not feasible. Therefore, this paper primarily focuses on locating specific household items or personal belongings. Sensory augmentation is a broad AT research area, including using visual, haptic/tactile (sense of touch), and auditory feedback as a mode of augmentation; this paper focuses on using auditory sensory augmentation for people with BLV. In this case, visual sensory augmentation methods are irrelevant to our end-user group. Haptic or tactile sensory augmentation is another popular AT form for BLV users. Past works have successfully translated visual information into wearable (helmets, belts, gloves, and smart canes) haptic/tactile feedback [12–15]. The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16]. Directional sense can only be achieved through sensory-motor coupling (relation between motor action and sensory feedback) and requires a higher level of physical and cognitive effect to gauge precise/complex spatial information. In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19]. Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22]. Many devices, both technological and mechanical in design, exist to address the various ADL needs of people who are BLV via sensory substitution and augmentation. Sensory substitution refers to converting an impaired sensory modality into another perceivable form to the user [23]. Sensory augmentation furthers this concept by using technology to augment or adapt sensory feedback to enhance the fidelity of information presented to the user [24]. The long cane is a good example of a sensory augmentation device that provides tactile sensory feedback as a mobility aid for independent travel [1]. Other more technologically-complicated examples are screen readers (text to speech), Braille displays (print to braille feedback) [25], handheld sonar devices (visual to auditory and tactile feedback) [6], and smartphone applications (Be My Eyes [26] and Seeing AI [27], visual/text to auditory feedback). The recent improvements in augmented reality (AR), wearable camera/sensor technology, and deep learning-based computer vision have led to the emergence of smart glasses as versatile and multifunctional AT [28, 29]. These systems tend to fuse a camera, Inertial Measurement Unit (IMU), microphone, Global Positioning System (GPS) and depth sensing data to provide services such as rendering environment/object/text/person as computer-synthesized speech, voice recognition control (for internet search or booking services), navigation, and voice/video communication (social call or contacting a human guide) [30, 31]. These smart glasses have many important benefits, such as being hands-free, inconspicuous in appearance, and multifunctional. Many smart glasses systems [30, 31] leverages artificial technology and speech audio feedback to deliver a reliable and robust real-time solution to aiding real-world navigation [32]. Smart glasses technology is growing in popularity and slowly being adopted within the BLV community. Inspired by human echolocation training [4, 33], we explored the concept of “acoustic touch”, coined by Jin et al. [9], which involves the use of head scanning and the activation of auditory icons (instead of computer-synthesized speech [31]) as objects appear within a defined field of view to convey the identity and location of various items to the user. Acoustic touch builds on previous research and incorporates sensory-motor coupling via head scanning and auditory icon sonification to enhance the spatial perception of close items [9]. We propose that the technique of acoustic touch offers these three advantages:
What the sources say
- Partly supportedcites [16] Kärcher SM, Fenzlaff S, Hartmann D, Nagel SK, König P. Sensory Augmentation for the Blind. Frontiers in Human Neuroscience. 2012; 6: 6– 37. doi: 10.3389/fnhum.2012.00037 22403535 PMC3290767 sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) says integrating tactile sensory information requires learning complex perceptual skills, and the Discussion (page unavailable) links training and prior cognitive grounding to spatial perception. The paper does not establish that significant training is the main challenge across haptic augmentation or that it produces semantic awareness of the surroundings; it notes that a subject with prior spatial grounding could benefit immediately. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22].”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 1 Introduction Assistive technology (AT) is an extensive research field that involves designing technologies to enable an individual with a sensory disability to overcome various physical, social, infrastructural and accessibility barriers in their activities of daily living (ADL) [1]. One central area within this field is developing assistive technology for people who are blind or have low vision (BLV). According to the World Health Organisation, it is estimated that blindness affects approximately 39 million people and, additionally, 246 million people have low vision [2]. Vision can be thought of as a primary and integrated sense, and it carries an important modality. The absence of vision affects many ADLs, and the lack of access to visual information impacts one’s ability to participate in a range of life and community activities [3]. Considerable research is dedicated to exploring the capability of AT [1]. With specialized Orientation and Mobility (O&M) training [4, 5] to assist people who are BLV or who have low vision. The common goal for AT and specialist training for people who are BLV is to achieve equitable access to facilitate independent living, individual career choices, and provide the ability to independently perform ADL, including navigation/travel [6], hazard detection/avoidance [7, 8], locating specific household items or personal belongings [9], reading/writing [10], and engaging in social interactions (including using social media) [11]. Human performance is a complex integration of sensory and motor systems, and exploring every ADL outlined is not feasible. Therefore, this paper primarily focuses on locating specific household items or personal belongings. Sensory augmentation is a broad AT research area, including using visual, haptic/tactile (sense of touch), and auditory feedback as a mode of augmentation; this paper focuses on using auditory sensory augmentation for people with BLV. In this case, visual sensory augmentation methods are irrelevant to our end-user group. Haptic or tactile sensory augmentation is another popular AT form for BLV users. Past works have successfully translated visual information into wearable (helmets, belts, gloves, and smart canes) haptic/tactile feedback [12–15]. The main challenge in haptic/tactile sensory augmentation is the significant training required to achieve a good spatial and semantic awareness of the user’s surroundings [16]. Directional sense can only be achieved through sensory-motor coupling (relation between motor action and sensory feedback) and requires a higher level of physical and cognitive effect to gauge precise/complex spatial information. In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19]. Using techniques such as computer-synthesized speech (e.g., object name and direction), spatial audio, head rotation, and modulated audio (pitch and amplitude modulation), past works have found success in using auditory sensory augmentation to provide users with accurate spatial information of their surrounding environment with very little training [15, 19–22]. Many devices, both technological and mechanical in design, exist to address the various ADL needs of people who are BLV via sensory substitution and augmentation. Sensory substitution refers to converting an impaired sensory modality into another perceivable form to the user [23]. Sensory augmentation furthers this concept by using technology to augment or adapt sensory feedback to enhance the fidelity of information presented to the user [24]. The long cane is a good example of a sensory augmentation device that provides tactile sensory feedback as a mobility aid for independent travel [1]. Other more technologically-complicated examples are screen readers (text to speech), Braille displays (print to braille feedback) [25], handheld sonar devices (visual to auditory and tactile feedback) [6], and smartphone applications (Be My Eyes [26] and Seeing AI [27], visual/text to auditory feedback). The recent improvements in augmented reality (AR), wearable camera/sensor technology, and deep learning-based computer vision have led to the emergence of smart glasses as versatile and multifunctional AT [28, 29]. These systems tend to fuse a camera, Inertial Measurement Unit (IMU), microphone, Global Positioning System (GPS) and depth sensing data to provide services such as rendering environment/object/text/person as computer-synthesized speech, voice recognition control (for internet search or booking services), navigation, and voice/video communication (social call or contacting a human guide) [30, 31]. These smart glasses have many important benefits, such as being hands-free, inconspicuous in appearance, and multifunctional. Many smart glasses systems [30, 31] leverages artificial technology and speech audio feedback to deliver a reliable and robust real-time solution to aiding real-world navigation [32]. Smart glasses technology is growing in popularity and slowly being adopted within the BLV community. Inspired by human echolocation training [4, 33], we explored the concept of “acoustic touch”, coined by Jin et al. [9], which involves the use of head scanning and the activation of auditory icons (instead of computer-synthesized speech [31]) as objects appear within a defined field of view to convey the identity and location of various items to the user. Acoustic touch builds on previous research and incorporates sensory-motor coupling via head scanning and auditory icon sonification to enhance the spatial perception of close items [9]. We propose that the technique of acoustic touch offers these three advantages:
What the sources say
- Not checked — no full text of the sourcecites [15] Lahav O, Mioduser D. Construction of cognitive maps of unknown spaces using a multi-sensory virtual environment for people who are blind. Computers in Human Behavior. 2008; 24(3): 1139– 1155. doi: 10.1016/j.chb.2007.04.003 source
- Not checked — no full text of the sourcecites [19] Papadopoulos K, Koustriava E, Barouti M. Cognitive maps of individuals with blindness for familiar and unfamiliar spaces: Construction through audio-tactile maps and walked experience. Computers in Human Behavior. 2017; 75: 376– 384. doi: 10.1016/j.chb.2017.04.057 source
- Partly supportedcites [20] Abboud S, Hanassy S, Levy-Tzedek S, Maidenbaum S, Amedi A. EyeMusic: Introducing a “visual” colorful experience for the blind using auditory sensory substitution. Restorative Neurology and Neuroscience. 2014; 32(2): 247– 257. doi: 10.3233/RNN-130338 24398719 sourceExperimental — accuracy not yet measured
The Introduction reports that users of sensory-substitution devices can localize stimuli, navigate, and avoid obstacles. But the Behavioral experiment tested identification of simple shapes and colors after 2–3 hours of training, not spatial awareness of the surrounding environment; the Discussion says locating real objects was observed only anecdotally after more training. The source therefore supports the spatial-information component, but not the claim that it was achieved with very little training. (LLM, conf=0.88)
- Partly supportedcites [21] Neugebauer A, Rifai K, Getzlaff M, Wahl S. Navigation aid for blind persons by visual-to-auditory sensory substitution: A pilot study. PLOS ONE. 2020; 15(8): e0237344. doi: 10.1371/journal.pone.0237344 32818953 PMC7446825 sourceExperimental — accuracy not yet measured
The Conclusion reports that blind participants navigated with the auditory device immediately after instructions and recognized 3D objects after limited use. But the tasks used a simulated environment synchronized with real-world objects; the Discussion says the device could not yet capture unmarked real-world obstacles. The source therefore supports success with little training in the test setting, not accurate information about a user's surroundings generally. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites [22] Chang CHS, Kuo CC, Hou HT, Koe JJY. Design and evaluation of a multi-sensory scaffolding gamification science course with mobile technology for learners with total blindness. Computers in Human Behavior. 2022; 128: 107085. doi: 10.1016/j.chb.2021.107085 source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The blindfolded sighted participants were significantly lower in success rate in the FAD condition compared to the Clock (Z = 2.21, p = 0.027, and r = 0.83) and Speaker condition (Z = 2.21, p = 0.027, and r = 0.83).”
§ An investigation into the effectiveness of using acoustic touch to assist people who are blind > 3 Results > 3.1 Seated task performance
plainn = 5 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
70 more claims
Create a free account to see all 124 claims and to verify the 4 references still open.
- Probably not a substantial claim. 32
- It may state two separate claims as one. 26
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = 2.21reported p = .027 · recomputed p = .027
“(Z = 2.21, p = .027”
- Consistentz = 2.2reported p = .028 · recomputed p = .028
“(Z = 2.20, p = .028”
- Consistentz = 2.03reported p = .043 · recomputed p = .042
“(Z = 2.03, p = .043”
- Consistentz = 2.2reported p = .028 · recomputed p = .028
“Z = 2.20, p = .028”
- Consistentz = 1.99reported p = .046 · recomputed p = .047
“Z = 1.99, p = .046”
5 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Lecture Notes in Computer Science 1205
(1999) · Industrial Robot: An International Journal · doi:10.1108/ir.1999.04926fae.001 · Cited by 38,662
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C. L., & Dollár, P. (2014) · arXiv · doi:10.48550/arxiv.1405.0312 · Cited by 2,579
Sensory substitution and the human-machine interface
Bach-y-Rita, P., & W. Kercel, S. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.10.013 · Cited by 858
Visual Prosody and Speech Intelligibility: Head Movement Improves Auditory Speech Perception
Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., & Vatikiotis-Bateson, E. (2004) · Psychological Science · doi:10.1111/j.0963-7214.2004.01502010.x · Cited by 482
Bhowmick, A., & Hazarika, S. M. (2017) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-016-0235-6 · Cited by 271
A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching
Gehrig, S. K., Eberli, F., & Meyer, T. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-04667-4_14 · Cited by 270
A summary of research investigating echolocation abilities of blind and sighted humans
Kolarik, A. J., Cirstea, S., Pardhan, S., & Moore, B. C. J. (2014) · Hearing Research · doi:10.1016/j.heares.2014.01.010 · Cited by 270
Static and Dynamic Hand Gesture Recognition in Depth Data Using Dynamic Time Warping
Plouffe, G., & Cretu, A.-M. (2016) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2015.2498560 · Cited by 266
Haptic wearables as sensory replacement, sensory augmentation and trainer – a review
Shull, P. B., & Damian, D. D. (2015) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-015-0055-z · Cited by 245
Validation of the Empatica E4 wristband
McCarthy, C., Pradhan, N., Redpath, C., & Adler, A. (2016) · 2016 IEEE EMBS International Student Conference (ISC) · doi:10.1109/embsisc.2016.7508621 · Cited by 235
Blind Navigation and the Role of Technology
Giudice, N. A., & Legge, G. E. (2008) · The Engineering Handbook of Smart Technology for Aging, Disability, and Independence · doi:10.1002/9780470379424.ch25 · Cited by 232
Directional sensitivity of sound-pressure levels in the human ear canal
Middlebrooks, J. C., Makous, J. C., & Green, D. M. (1989) · The Journal of the Acoustical Society of America · doi:10.1121/1.398224 · Cited by 222
Abboud, S., Hanassy, S., Levy-Tzedek, S., Maidenbaum, S., & Amedi, A. (2014) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-130338 · Cited by 199
Weighted Least Squares Fitting Using Ordinary Least Squares Algorithms
Kiers, H. A. L. (1997) · Psychometrika · doi:10.1007/bf02295279 · Cited by 199
Tactile Spatial Acuity Enhancement in Blindness: Evidence for Experience-Dependent Mechanisms
Wong, M., Gnanakumaran, V., & Goldreich, D. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6461-10.2011 · Cited by 183
How Blind People Interact with Visual Content on Social Networking Services
Voykinska, V., Azenkot, S., Wu, S., & Leshed, G. (2016) · Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing · doi:10.1145/2818048.2820013 · Cited by 174
On the relative complexity of active vs. passive visual search
Tsotsos, J. K. (1992) · International Journal of Computer Vision · doi:10.1007/bf00128132 · Cited by 164
Smart Glass System Using Deep Learning for the Blind and Visually Impaired
Mukhiddinov, M., & Cho, J. (2021) · Electronics · doi:10.3390/electronics10222756 · Cited by 163
Effects of anxiety sensitivity on the response to hyperventilation.
Holloway, W., & McNally, R. J. (1987) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.96.4.330 · Cited by 161
Perception of object length by sound
Carello, C., Anderson, K. L., & Kunkler-Peck, A. J. (1998) · Psychological Science · doi:10.1111/1467-9280.00040 · Cited by 153
Psychological stress and body temperature changes in humans
Marazziti, D., Di Muro, A., & Castrogiovanni, P. (1992) · Physiology & Behavior · doi:10.1016/0031-9384(92)90290-i · Cited by 139
Decoding the direction of auditory motion in blind humans
Wolbers, T., Zahorik, P., & Giudice, N. A. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.04.266 · Cited by 131
Slade, P., Tambe, A., & Kochenderfer, M. J. (2021) · Science Robotics · doi:10.1126/scirobotics.abg6594 · Cited by 118
White, C., & Caird, J. K. (2010) · Accident Analysis & Prevention · doi:10.1016/j.aap.2010.05.003 · Cited by 115
Postma, A., Zuidhoek, S., Noordzij, M. L., & Kappers, A. M. L. (2007) · Perception · doi:10.1068/p5441 · Cited by 110
From science to technology: Orientation and mobility in blind children and adults
Cuturi, L. F., Aggius-Vella, E., Campus, C., Parmiggiani, A., & Gori, M. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.08.019 · Cited by 107
Psychometric properties of the Zephyr bioharness device: a systematic review
Nazari, G., Bobos, P., MacDermid, J. C., Sinden, K. E., Richardson, J., & Tang, A. (2018) · BMC Sports Science, Medicine and Rehabilitation · doi:10.1186/s13102-018-0094-4 · Cited by 101
Sensory Augmentation for the Blind
Kärcher, S. M., Fenzlaff, S., Hartmann, D., Nagel, S. K., & König, P. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00037 · Cited by 93
Lahav, O., & Mioduser, D. (2008) · Computers in Human Behavior · doi:10.1016/j.chb.2007.04.003 · Cited by 89
Changes in breathing during observation of effortful actions
Paccalin, C., & Jeannerod, M. (2000) · Brain Research · doi:10.1016/s0006-8993(00)02145-4 · Cited by 84
Echolocation may have real-life advantages for blind people: an analysis of survey data
Thaler, L. (2013) · Frontiers in Physiology · doi:10.3389/fphys.2013.00098 · Cited by 64
Low cost ultrasonic smart glasses for blind
Agarwal, R., Ladha, N., Agarwal, M., et al. (2017) · IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) · doi:10.1109/iemcon.2017.8117194 · Cited by 64
Self-citationEvaluation and Comparison of Artificial Intelligence Vision Aids: Orcam MyEye 1 and Seeing AI
Granquist, C., Sun, S. Y., Montezuma, S. R., Tran, T. M., Gage, R., & Legge, G. E. (2021) · Journal of Visual Impairment & Blindness · doi:10.1177/0145482x211027492 · Cited by 64
Trillo, A. H., & Dickinson, C. (2012) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.12-9580 · Cited by 61
An Implementation of an Intelligent Assistance System for Visually Impaired/Blind People
Chen, L.-B., Su, J.-P., Chen, M.-C., Chang, W., Yang, C.-H., & Sie, C.-Y. (2019) · International Conference on Consumer Electronics · doi:10.1109/icce.2019.8661943 · Cited by 61
Psychophysical customization of directional transfer functions for virtual sound localization
Middlebrooks, J. C., Macpherson, E. A., & Onsan, Z. A. (2000) · The Journal of the Acoustical Society of America · doi:10.1121/1.1322026 · Cited by 59
Remote Assistance for Blind Users in Daily Life: A Survey about Be My Eyes
Avila, M., Wolf, K., Brock, A., & Henze, N. (2016) · Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments · doi:10.1145/2910674.2935839 · Cited by 59
Perception of Speech and Sound
Kollmeier, B., Brand, T., & Meyer, B. (2008) · Springer Handbooks · doi:10.1007/978-3-540-49127-9_4 · Cited by 56
Assistive Text Reading from Complex Background for Blind Persons
Yi, C., & Tian, Y. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-29364-1_2 · Cited by 55
Deep Learning Assisted Smart Glasses as Educational Aid for Visually Challenged Students
AlSaid, H., AlKhatib, L., AlOraidh, A., AlHaidar, S., & Bashar, A. (2019) · International Conference on New Trends in Computing Sciences (ICTCS) · doi:10.1109/ictcs.2019.8923044 · Cited by 43
Navigation aid for blind persons by visual-to-auditory sensory substitution: A pilot study
Neugebauer, A., Rifai, K., Getzlaff, M., & Wahl, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0237344 · Cited by 41
Survey Design for Visually Impaired and Blind People
Kaczmirek, L., & Wolff, K. G. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-73279-2_41 · Cited by 38
Papadopoulos, K., Koustriava, E., & Barouti, M. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.04.057 · Cited by 37
Sophie Chang, C.-H., Kuo, C.-C., Hou, H.-T., & Ying Koe, J. J. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2021.107085 · Cited by 37
The Miniguide: A New Electronic Travel Device
Hill, J., & Black, J. (2003) · Journal of Visual Impairment & Blindness · doi:10.1177/0145482x0309701010 · Cited by 31
Sensory Augmentation with Distal Touch: The Tactile Helmet Project
Bertram, C., Evans, M. H., Javaid, M., Stafford, T., & Prescott, T. J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39802-5_3 · Cited by 29
Sensory augmentation: integration of an auditory compass signal into human perception of space
Schumann, F., & O’Regan, J. K. (2017) · Scientific Reports · doi:10.1038/srep42197 · Cited by 25
Chen, Z., Cao, H., Wang, H., Xu, F., Kostakos, V., & Li, Y. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3411812 · Cited by 20
Sorokowska, A., Hummel, T. R., & Oleszkiewicz, A. (2020) · Neuroscience · doi:10.1016/j.neuroscience.2020.05.033 · Cited by 18
Towards Outdoor Navigation System for Visually Impaired People using YOLOv5
Chandna, S., & Singhal, A. (2022) · 2022 12th International Conference on Cloud Computing, Data Science & Engineering (Confluence) · doi:10.1109/confluence52989.2022.9734204 · Cited by 17
Comparing interaction techniques to help blind people explore maps on small tactile devices
Simonnet, M., Brock, A. M., Serpa, A., Oriola, B., & Jouffrais, C. (2019) · Multimodal Technologies and Interaction · doi:10.3390/mti3020027 · Cited by 16
Using Dynamic Audio Feedback to Support Peripersonal Reaching in Young Visually Impaired People
Wilson, G., & Brewster, S. A. (2016) · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2982142.2982160 · Cited by 16
Chen, K., Huang, Y., Chen, Y., Zhong, H., Lin, L., Wang, L., & Wu, K. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550282 · Cited by 12
Nreal: Ready-to-wear Mixed Reality Glasses
Xu, C. (2021) · SPIE AVR21 Industry Talks II · doi:10.1117/12.2597454 · Cited by 8
(2021) · PubMed · Cited by 5
Acoustic touch: An auditory sensing paradigm to support close reaching for people who are blindPreprint
Jin, C. T., Bell, J.-A. R., Deverell, L., Gates, F., Gorodo, I., Hossain, S. N., Lin, C., Melencio, M. V., Nguyen, M., Nguyen, V. A., et al. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/qse5h · Cited by 4
Self-citationHoward, M. (2020) · Information Technology and Libraries · doi:10.6017/ital.v39i3.12637 · Cited by 2
Mishra, A., Raj, N., Bodhe, S., & Bajwa, G. (2022) · 2022 IEEE 23rd International Conference on Information Reuse and Integration for Data Science (IRI) · doi:10.1109/iri54793.2022.00067 · Cited by 1
Research works printed but not linked: 1
- Dingler T, Lindsay J, Walker BN, et al. Learnability of sound cues for environmental features: Auditory icons, earcons, spearcons, and speech. In: Proceedings of the 14th International Conference on Auditory Display, Paris, France; 2008. p. 1–6. Available from: http://www.icad.org/Proceedings/2008/DinglerLindsay2008.pdf.
Cited by
A Human-Centric Metaverse Enabled by Brain-Computer Interface: A Survey
Zhu, H. Y., Hieu, N. Q., Hoang, D. T., Nguyen, D. N., & Lin, C. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3387124 · Cited by 100
“BCI can also be integrated with other assistive technologies [25] or input modalities [26] to create multimodal interfaces [27].”
BackgroundSelf-citationLimweshasin, N., Castro, I. A. A., Korposh, S., Morgan, S. P., Hayes‐Gill, B., Faghy, M. A., & Correia, R. G. (2024) · Sensors · doi:10.3390/s24237476 · Cited by 10
Nguyen, M. T. D., Zhu, H. Y., Sun, H., et al. (2024) · doi:10.1145/3688862.3689115 · Cited by 6
“The human brain utilises binaural hearing to localise sounds both passively and actively [9, 19, 23].”
Self-citationMinton, A. G., Zhu, H. Y., Chen, H., Wang, Y., Zhuang, Z., Notaro, G. M., Galvan-Garza, R. C., Allen, J. P., Ziegler, M. D., & Lin, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713402 · Cited by 5
Self-citationAtes, Ö., Pandey, G., Gousiopoulos, A., & Soldatos, T. G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1424371 · Cited by 3
Metadata
- Authors
- Zhu, H. Y., Hossain, S. N., Jin, C., Singh, A. K., Nguyen, M. T. D., Deverell, L., Nguyen, V., Gates, F. S., Fernandez, I. G., Melencio, M. V., Bell, J.-A. R., & Lin, C.-T.
- Affiliations
- University of Technology Sydney (AU)
- The University of Sydney (AU)
- Corresponding author
- Zhu, Howe Yuan
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-10-25
- Volume
- 18
- Issue
- 10
- Pages
- e0290431
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC10599575
- OpenAlex
- W4387950336
- Semantic Scholar
- 2b33b37418485873173142769007a23aa1a99db1
- Cited by
- 16
- References
- 57
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 6
- FWCI
- 2.39
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Australian Cooperative Research Centres Projects (CRCPXI000007)
- the ARIA research
- the University of Technology Sydney
- the University of Sydney
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Keywords
- assistive technology for the blind, low vision, spatial audio, sonification, auditory icons, object recognition, sensory augmentation, mental workload
- MeSH terms
- Smart Glasses, Acoustics, Blindness, Humans, Speech, Vision, Ocular, Touch Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.