CoChat
Sign Up
Cited by 15Neuroscience and Neural EngineeringTactile and Sensory InteractionsVisual perception and processing mechanisms

Testing vision is not testing for vision

Open access
Peli, Eli·2020·Translational Vision Science & Technology
DOI 10.1167/tvst.9.13.32PMC PMC7757632
Open this paper in CoChat — free account

Abstract

Visual prostheses aim to restore, at least to some extent, vision that leads to the type of perception available for sighted patients. Their effectiveness is almost always evaluated using clinical tests of vision. Clinical vision tests are designed to measure the limits of parameters of a functioning visual system. I argue here that these tests are rarely suited to determine the ability of prosthetic devices and other therapies to restore vision. This paper describes and explains many limitations of these evaluations. Prosthetic vision testing often makes use of multiple-alternative forced-choice (MAFC) procedures. Although these paradigms are suitable for many studies, they are frequently problematic in vision restoration evaluation. Two main types of problems are identified: (1) where nuisance variables provide spurious cues that can be learned in repeated training, which is common in prosthetic vision, and thus defeat the purpose of the test; and (2) even though a test is properly designed and performed, it may not actually measure what the researchers believe, and thus the interpretation of results is wrong. Examples for both types of problems are presented. Additional problems arise from confounding factors in the administration of tests are pointed as limitations of current device evaluation. For example, head tracing of magnified objects enlarged to compensate for the system's low resolution, in distinction from the scanning head (camera) movements with which users of prosthetic devices expand the limited field of view. Because of these problems, the ability to perform satisfactorily on the clinical tests is necessary but insufficient to prove vision restoration, therefore, additional tests are needed. I propose some directions to pursue in such testing. Translational Relevance: Numerous prosthetic devices are being developed and introduced to the market. Proving the utility of these devices is crucial for regulatory and even for post market acceptance, which so far has largely failed, in my opinion. Potential reasons for the failures despite success in regulatory testing and directions for designing improved testing are provided. It is hoped that improved testing will guide improved designs of future prosthetic systems and other vision restoration approaches.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1167/tvst.9.13.32

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Because of the multi-alternative forced choice (MAFC) paradigm, clinical and psychophysics vision tests may not measure what researchers intended, either because nuisance variables may provide spurious cues that can be learned in repeated training or because the tests can be passed without form vision.

    “The main reason for this is the nature of the multi-alternative forced choice (MAFC) paradigm that is typically used to administer these tests. As such, they may not measure what the researchers intended, either because nuisance variables may provide spurious cues that can be learned in repeated training or because the tests can be passed without form vision (Peli, 2020).”

    From Towards a <i>Smart Bionic Eye</i> : AI-powered artificial vision for the treatment of incurable blindness (2022) · cites this paper as Peli, 2020

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Vision tests typically used in clinics and psychophysics laboratories are not designed to test the ability of prosthetic devices to restore vision.

    “Another concern is that the vision tests typically used in clinics and psychophysics laboratories (e.g., perimetry, acuity, contrast sensitivity, orientation discrimination) are not designed to test the ability of prosthetic devices to restore vision (Peli, 2020).”

    From Towards a <i>Smart Bionic Eye</i> : AI-powered artificial vision for the treatment of incurable blindness (2022) · cites this paper as Peli, 2020

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [39]Stingl K. How a subject perceived his wife's face. 2013 Available at https://www.youtube.com/watch?v=40JbGXTbSOQ. Accessed April 8, 2020. This research was published in the journal Proceedings Biological Sciences, in the paper ‘Artificial vision with wirelessly powered subretinal electronic implant alpha-IMS’ by Stingl et al. The DOI link for the article is: 10.1098/rspb.2013.0077. PMC3619489 23427175 · DOI · Paper page

Cited by

Sorted by citations

Related

Authors
Peli, E.
Affiliations
Schepens Eye Research Institute of Mass Eye & Ear, Department of Ophthalmology, Harvard Medical School, Boston, MA, USA
Venue
Translational Vision Science & Technology
Publisher
Association for Research in Vision and Ophthalmology (ARVO)
Published
2020-12-18
Volume
9
Issue
13
Pages
32
ISSN
2164-2591 (ISSN-L)
DOI
10.1167/tvst.9.13.32
PMC
PMC7757632
OpenAlex
W3117015481
Cited by
14
References
41
Citations by year
2026: 3 · 2025: 1 · 2024: 2 · 2023: 2 · 2022: 5 · 2021: 1
FWCI
0.92
Type
journal article
Language
en
Open access
Licence
Locations
  • Translational Vision Science & Technology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
National Eye Institute (P30EY003790)
Topics
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neural Engineering
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
Keywords
retinal prosthesis, vision restoration, nuisance parameters, confounding factors, device evaluation, regulatory testing, test validity
MeSH terms
Humans, Movement, Vision, Ocular, Vision Tests, Visual Prosthesis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review