Policy-Driven, Multimodal Deep Learning for Predicting Visual Fields from the Optic Disc and OCT Imaging
Open accessAbstract
Purpose To develop and validate a deep learning (DL) system for predicting each point on visual fields (VFs) from disc and OCT imaging and derive a structure-function mapping. Design Retrospective, cross-sectional database study. Participants A total of 6437 patients undergoing routine care for glaucoma in 3 clinical sites in the United Kingdom. Methods OCT and infrared reflectance (IR) optic disc imaging were paired with the closest VF within 7 days. EfficientNet B2 was used to train 2 single-modality DL models to predict each of the 52 sensitivity points on the 24-2 VF pattern. A policy DL model was designed and trained to fuse the 2 model predictions. Main outcome measures Pointwise mean absolute error (PMAE). Results A total of 5078 imaging scans to VF pairs were used as a held-out test set to measure the final performance. The improvement in PMAE with the policy model was 0.485 (0.438, 0.533) decibels (dB) compared with the IR image of the disc alone and 0.060 (0.047, 0.073) dB with to the OCT alone. The improvement with the policy fusion model was statistically significant (P < 0.0001). Occlusion masking shows that the DL models learned the correct structure-function mapping in a data-driven, feature agnostic fashion. Conclusions The multimodal, policy DL model performed the best; it provided explainable maps of its confidence in fusing data from single modalities and provides a pathway for probing the structure-function relationship in glaucoma.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Optical coherence tomography and infrared reflectance optic disc imaging combined better predict visual field maps compared to using either modality alone.
“Optical coherence tomography and infrared reflectance optic disc imaging have been combined to better predict visual field maps compared to using either of those modalities alone 131”
From Multimodal biomedical AI (2022) · cites this paper as [131]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Pathophysiology and Treatment of Glaucoma: A Review
Weinreb, R. N., Aung, T., & Medeiros, F. A. (2014) · JAMA · doi:10.1001/jama.2014.3192 · Cited by 4,210 · see citing works
Garway‐Heath, D. F., Poinoosawmy, D., Fitzke, F. W., et al. (2000) · Ophthalmology · doi:10.1016/s0161-6420(00)00284-0 · Cited by 737 · see citing works
A framework for comparing structural and functional measures of glaucomatous damage
Hood, D. C., & Kardon, R. H. (2007) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2007.08.001 · Cited by 708 · see citing works
OCT for glaucoma diagnosis, screening and detection of glaucoma progression
Bussel, I. I., Wollstein, G., & Schuman, J. S. (2013) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2013-304326 · Cited by 443 · see citing works
Artes, P. H., Iwase, A., Ohno, Y., et al. (2002) · PubMed · Cited by 360 · see citing works
Drasdo, N., Millican, C. L., Katholi, C. R., & Curcio, C. A. (2007) · Vision Research · doi:10.1016/j.visres.2007.01.007 · Cited by 283 · see citing works
The Onset and Evolution of Glaucomatous Visual Field Defects
Hart, W. M., & BECKER, B. A. (1982) · Ophthalmology · doi:10.1016/s0161-6420(82)34798-3 · Cited by 253 · see citing works
Bengtsson, B., & Heijl, A. (2000) · PubMed · Cited by 202 · see citing works
Evidence-based Criteria for Assessment of Visual Field Reliability
Yohannan, J., Wang, J., Brown, J. C., et al. (2017) · Ophthalmology · doi:10.1016/j.ophtha.2017.04.035 · Cited by 195 · see citing works
Christopher, M. A., Bowd, C., Belghith, A., Goldbaum, M. H., Weinreb, R. N., Fazio, M. A., Girkin, C. A., Liebmann, J. M., & Zangwill, L. M. (2019) · Ophthalmology · doi:10.1016/j.ophtha.2019.09.036 · Cited by 181 · see citing works
Jansonius, N. M., Nevalainen, J., Selig, B., Zangwill, L. M., Sample, P. A., Budde, W. M., Jonas, J. B., Lagrèze, W. A., Airaksinen, P. J., Vonthein, R., et al. (2009) · Vision Research · doi:10.1016/j.visres.2009.04.029 · Cited by 155 · see citing works
Turpin, A. H., McKendrick, A. M., Johnson, C. A., et al. (2003) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.03-0023 · Cited by 137 · see citing works
Quantitative evaluation of the optic nerve head in early glaucoma
Garway‐Heath, D. F., & Hitchings, R. A. (1998) · British Journal of Ophthalmology · doi:10.1136/bjo.82.4.352 · Cited by 120 · see citing works
Bengtsson, B., & Heijl, A. (2000) · American Journal of Ophthalmology · doi:10.1016/s0002-9394(00)00758-3 · Cited by 111 · see citing works
Jansonius, N. M., Schiefer, J., Nevalainen, J., et al. (2012) · Experimental Eye Research · doi:10.1016/j.exer.2012.10.008 · Cited by 107 · see citing works
Russell, R. A., Malik, R., Chauhan, B. C., et al. (2012) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.11-7976 · Cited by 97 · see citing works
van der Schoot, J., Vermeer, K. A., de Boer, J. F., & Lemij, H. G. (2012) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.11-8436 · Cited by 91 · see citing works
Lamparter, J., Russell, R. A., Zhu, H., et al. (2013) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.13-11902 · Cited by 78 · see citing works
Hood, D. C., Fortune, B. A., Mavrommatis, M. A., et al. (2015) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.15-17259 · Cited by 75 · see citing works
Predicting Visual Function from the Measurements of Retinal Nerve Fiber Layer Structure
Zhu, H., Crabb, D. P., Schlottmann, P. G., et al. (2010) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.10-5239 · Cited by 72 · see citing works
Self-citationArtificial Intelligence Mapping of Structure to Function in Glaucoma
Mariottoni, E. B., Datta, S., Dov, D., et al. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.2.19 · Cited by 72 · see citing works
A deep learning approach to predict visual field using optical coherence tomography
Park, K., Kim, J., & Lee, J. W. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234902 · Cited by 61 · see citing works
Denniss, J., McKendrick, A. M., & Turpin, A. H. (2012) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.12-9657 · Cited by 60 · see citing works
The visualFields package: A tool for analysis and visualization of visual fields
Marín‐Franch, I., & Swanson, W. H. (2013) · Journal of Vision · doi:10.1167/13.4.10 · Cited by 52 · see citing works
Guo, Z., Kwon, Y. H., Lee, K., et al. (2017) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.17-21832 · Cited by 49 · see citing works
Hashimoto, Y., Asaoka, R., Kiwaki, T., et al. (2020) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2019-315600 · Cited by 49 · see citing works
Turpin, A. H., Sampson, G. P., & McKendrick, A. M. (2009) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.08-2492 · Cited by 48 · see citing works
Yu, H., Maetschke, S., Antony, B. J., et al. (2020) · Ophthalmology Glaucoma · doi:10.1016/j.ogla.2020.07.002 · Cited by 47 · see citing works
Denniss, J., McKendrick, A. M., & Turpin, A. H. (2013) · Translational Vision Science & Technology · doi:10.1167/tvst.2.4.3 · Cited by 35 · see citing works
A Perimetric Test Procedure That Uses Structural Information
Ganeshrao, S. B., McKendrick, A. M., Denniss, J., et al. (2014) · Optometry and Vision Science · doi:10.1097/opx.0000000000000447 · Cited by 27 · see citing works
Shin, J., Kim, S. J., Kim, J., et al. (2021) · Translational Vision Science & Technology · doi:10.1167/tvst.10.7.4 · Cited by 27 · see citing works
Revisiting the Drasdo Model: Implications for Structure-Function Analysis of the Macular Region
Montesano, G., Ometto, G., Hogg, R. E., et al. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.10.15 · Cited by 26 · see citing works
Self-citationImproving Visual Field Examination of the Macula Using Structural Information
Montesano, G., Rossetti, L. M., Allegrini, D., et al. (2018) · Translational Vision Science & Technology · doi:10.1167/tvst.7.6.36 · Cited by 23 · see citing works
Self-citationDatta, S., Mariottoni, E. B., Dov, D., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-91493-9 · Cited by 21 · see citing works
Qiu, K., Schiefer, J., Nevalainen, J., et al. (2015) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.15-17450 · Cited by 17 · see citing works
Estimating visual field loss from monoscopic optic disc photography using deep learning model
Lee, J., Kim, Y. W., Ha, A., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-78144-1 · Cited by 16 · see citing works
Wall, M., Lee, E. J., Wanzek, R. J., et al. (2020) · Journal of Glaucoma · doi:10.1097/ijg.0000000000001427 · Cited by 13 · see citing works
Asaoka, R., Murata, H., Matsuura, M., et al. (2020) · Ophthalmology Glaucoma · doi:10.1016/j.ogla.2020.01.001 · Cited by 13 · see citing works
Improving Personalized Structure to Function Mapping From Optic Nerve Head to Visual Field
Turpin, A. H., & McKendrick, A. M. (2021) · Translational Vision Science & Technology · doi:10.1167/tvst.10.1.19 · Cited by 13 · see citing works
Temporal visual field in glaucoma: a re-evaluation in the automated perimetry era
Pennebaker, G. E., & Stewart, W. C. (1992) · Graefe s Archive for Clinical and Experimental Ophthalmology · doi:10.1007/bf00164646 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [15]Jansonius NM, Nevalainen J, Selig B, A mathematical description of nerve fiber bundle trajectories and their variability in the human retina. Vision Res 2009; 49: 2157– 2163. 19539641 PMC2848177
- [27]Tan M, Le Q. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In: International Conference on Machine Learning. PMLR; 2019: 6105– 6114. Available at: [ Accessed June 17, 2021].
- [28]Dozat T Incorporating Nesterov Momentum into Adam. 2016. Available at: https://openreview.net/pdf?id=OM0jvwB8jlp57ZJjtNEZ [ Accessed June 17, 2021].
- [29]Jansonius NM, Schiefer J, Nevalainen J, A mathematical model for describing the retinal nerve fiber bundle trajectories in the human eye: average course, variability, and influence of refraction, optic disc size and optic disc position. Exp Eye Res 2012; 105: 70– 78. 23099334
Cited by
Sorted by citations
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365 · see citing works
- Authors
- Kihara, Y., Montesano, G., Chen, A., Amerasinghe, N., Dimitriou, C., Jacob, A., Chabi, A., Crabb, D. P., & Lee, A. Y.
- Affiliations
- University of Washington (US)
- Moorfields Eye Hospital NHS Foundation Trust (GB)
- University Hospital Southampton NHS Foundation Trust (GB)
- East Suffolk and North Essex NHS Foundation Trust (GB)
- Santen (United States) (US)
- City, University of London (GB)
- Corresponding author
- Lee, Aaron Y.
- ORCID
- Venue
- Ophthalmology
- Publisher
- Elsevier BV
- Published
- 2022-07
- Volume
- 129
- Issue
- 7
- Pages
- 781-791
- ISSN
- 0161-6420 (ISSN-L)
- PMC
- PMC9233104
- OpenAlex
- W4212944981
- Cited by
- 62
- References
- 40
- Citations by year
- 2026: 18 · 2025: 15 · 2024: 11 · 2023: 12 · 2022: 6
- FWCI
- 9.84
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Ophthalmology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- City Research Online (City University London) (acceptedVersion)
- Funders
- National Institute on Aging (U19AG066567)
- National Eye Institute (K23EY029246)
- Topics
- Health Sciences › Medicine › Ophthalmology › Glaucoma and retinal disorders
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- glaucoma, visual field prediction, optic disc imaging, infrared reflectance imaging, optical coherence tomography, multimodal deep learning, structure-function mapping, explainable AI
- MeSH terms
- Deep Learning, Glaucoma, Humans, Intraocular Pressure, Optic Disk, Optic Nerve Diseases, Retrospective Studies, Visual Fields, Tomography, Optical Coherence, Policy, Visual Field Tests
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
