Neurolight: A Deep Learning Neural Interface for Cortical Visual Prostheses
The DOI matches its Crossref record; 63 of the 65 references it cites were found in a registry or an index.
Abstract
Visual neuroprosthesis, that provide electrical stimulation along several sites of the human visual system, constitute a potential tool for vision restoration for the blind. Scientific and technological progress in the fields of neural engineering and artificial vision comes with new theories and tools that, along with the dawn of modern artificial intelligence, constitute a promising framework for the further development of neurotechnology. In the framework of the development of a Cortical Visual Neuroprosthesis for the blind (CORTIVIS), we are now facing the challenge of developing not only computationally powerful tools and flexible approaches that will allow us to provide some degree of functional vision to individuals who are profoundly blind. In this work, we propose a general neuroprosthesis framework composed of several task-oriented and visual encoding modules. We address the development and implementation of computational models of the firing rates of retinal ganglion cells and design a tool — Neurolight — that allows these models to be interfaced with intracortical microelectrodes in order to create electrical stimulation patterns that can evoke useful perceptions. In addition, the developed framework allows the deployment of a diverse array of state-of-the-art deep-learning techniques for task-oriented and general image pre-processing, such as semantic segmentation and object detection in our system’s pipeline. To the best of our knowledge, this constitutes the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array. We implement the complete pipeline, from obtaining a video stream to developing and deploying task-oriented deep-learning models and predictive models of retinal ganglion cells’ encoding of visual inputs under the control of a neurostimulation device able to send electrical train pulses to a microelectrode array implanted at the visual cortex.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Machine learning approaches, such as convolutional neural networks [189] and deep learning [190,191], could recognise multiple objects in complex scenes, helping navigation, obstacle avoidance and target object finding.”
From Advances in visual prostheses: engineering and biological challenges (2022) · cites this paper as [190]
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Deep learning in agriculture: A survey
Kamilaris, A., & Prenafeta-Boldú, F. X. (2018) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.02.016 · Cited by 5,133
Deep Brain Stimulation for Treatment-Resistant Depression
Mayberg, H. S., Lozano, A. M., Voon, V., McNeely, H. E., Seminowicz, D., Hamani, C., Schwalb, J. M., & Kennedy, S. H. (2005) · Neuron · doi:10.1016/j.neuron.2005.02.014 · Cited by 4,059
Self-citationBrain-computer interface technology: a review of the first international meeting
Wolpaw, J. R., Birbaumer, N., Heetderks, W. J., McFarland, D. J., Peckham, P. H., Schalk, G., Donchin, E., Quatrano, L. A., Robinson, C. J., & Vaughan, T. M. (2000) · IEEE Transactions on Rehabilitation Engineering · doi:10.1109/tre.2000.847807 · Cited by 2,324
Neuromorphic Silicon Neuron Circuits
Indiveri, G., Linares-Barranco, B., Hamilton, T. J., Schaik, A. V., Etienne-Cummings, R., Delbruck, T., Liu, S.-C., Dudek, P., Häfliger, P., Renaud, S., et al. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00073 · Cited by 1,821
Deep learning for smart manufacturing: Methods and applications
Wang, J., Ma, Y., Zhang, L., Gao, R. X., & Wu, D. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.01.003 · Cited by 1,740
Cochlear implants: A remarkable past and a brilliant future
Wilson, B. S., & Dorman, M. F. (2008) · Hearing Research · doi:10.1016/j.heares.2008.06.005 · Cited by 868
Translational principles of deep brain stimulation
Kringelbach, M. L., Jenkinson, N., Owen, S. L. F., & Aziz, T. Z. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2196 · Cited by 853
Brain-Controlled Interfaces: Movement Restoration with Neural Prosthetics
Schwartz, A. B., Cui, X. T., Weber, D. J., & Moran, D. W. (2006) · Neuron · doi:10.1016/j.neuron.2006.09.019 · Cited by 792
MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices
Chen, S., Liu, Y., Gao, X., & Han, Z. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-97909-0_46 · Cited by 771
Schmidt, E. M., Bak, M. J., Hambrecht, F. T., Kufta, C. V., O'Rourke, D. K., & Vallabhanath, P. (1996) · Brain · doi:10.1093/brain/119.2.507 · Cited by 672
Maynard, E. M., Nordhausen, C. T., & Normann, R. A. (1997) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/s0013-4694(96)95176-0 · Cited by 640
Histed, M. H., Bonin, V., & Reid, R. C. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.016 · Cited by 604
Artificial Vision for the Blind by Connecting a Television Camera to the Visual Cortex
Dobelle, W. H. (2000) · ASAIO Journal · doi:10.1097/00002480-200001000-00002 · Cited by 584
A Review of Organic and Inorganic Biomaterials for Neural Interfaces
Fattahi, P., Yang, G., Kim, G., & Abidian, M. R. (2014) · Advanced Materials · doi:10.1002/adma.201304496 · Cited by 580
Schwartz, A. B. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144233 · Cited by 559
Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems
Chicca, E., Stefanini, F., Bartolozzi, C., & Indiveri, G. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2313954 · Cited by 552
Fast and Slow Contrast Adaptation in Retinal Circuitry
Baccus, S. A., & Meister, M. (2002) · Neuron · doi:10.1016/s0896-6273(02)01050-4 · Cited by 515
Dobelle, W. H., & Mladejovsky, M. G. (1974) · The Journal of Physiology · doi:10.1113/jphysiol.1974.sp010766 · Cited by 468
The threshold for conscious report: Signal loss and response bias in visual and frontal cortex
van Vugt, B., Dagnino, B., Vartak, D., Safaai, H., Panzeri, S., Dehaene, S., & Roelfsema, P. R. (2018) · Science · doi:10.1126/science.aar7186 · Cited by 394
Weiland, J. D., Liu, W., & Humayun, M. S. (2005) · Annual Review of Biomedical Engineering · doi:10.1146/annurev.bioeng.7.060804.100435 · Cited by 371
Five-year safety and performance results from the argus II retinal prosthesis system clinical trial.
da Cruz, L., Dorn, J. D., Humayun, M. S., Dagnelie, G., Handa, J., Barale, P.-O., Sahel, J.-A., Stanga, P. E., Hafezi, F., Safran, A. B., et al. (2016) · Ophthalmology · doi:10.1016/j.ophtha.2016.06.049 · Cited by 359
Deep convolutional models improve predictions of macaque V1 responses to natural images
Cadena, S. A., Denfield, G. H., Walker, E. Y., Gatys, L. A., Tolias, A. S., Bethge, M., & Ecker, A. S. (2019) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006897 · Cited by 347
Davis, T. S., Wark, H. A. C., Hutchinson, D. T., Warren, D. J., O’Neill, K., Scheinblum, T., Clark, G. A., Normann, R. A., & Greger, B. (2016) · Journal of Neural Engineering · doi:10.1088/1741-2560/13/3/036001 · Cited by 314
Lewis, P. M., Ackland, H. M., Lowery, A. J., & Rosenfeld, J. V. (2015) · Brain Research · doi:10.1016/j.brainres.2014.11.020 · Cited by 228
Integration and Exploitation of Sensor Data in Smart Cities through Event-Driven Applications
Garcia Alvarez, M., Morales, J., & Kraak, M.-J. (2019) · Sensors · doi:10.3390/s19061372 · Cited by 184
Sengupta, A., Chaffiol, A., Macé, E., Caplette, R., Desrosiers, M., Lampič, M., Forster, V., Marre, O., Lin, J. Y., Sahel, J., et al. (2016) · EMBO Molecular Medicine · doi:10.15252/emmm.201505699 · Cited by 180
Cortical control of a tablet computer by people with paralysis
Nuyujukian, P., Albites Sanabria, J., Saab, J., Pandarinath, C., Jarosiewicz, B., Blabe, C. H., Franco, B., Mernoff, S. T., Eskandar, E. N., Simeral, J. D., et al. (2018) · PLOS ONE · doi:10.1371/journal.pone.0204566 · Cited by 170
A model for intracortical visual prosthesis research
Troyk, P., Bak, M., Berg, J., Bradley, D., Cogan, S., Erickson, R., Kufta, C., McCreery, D., Schmidt, E., & Towle, V. (2003) · Artificial Organs · doi:10.1046/j.1525-1594.2003.07308.x · Cited by 166
Toward the development of a cortically based visual neuroprosthesis
Normann, R. A., Greger, B. A., House, P., Romero, S. F., Pelayo, F., & Fernandez, E. (2009) · Journal of Neural Engineering · doi:10.1088/1741-2560/6/3/035001 · Cited by 143
Single Image Depth Estimation Trained via Depth From Defocus Cues
Gur, S., & Wolf, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00787 · Cited by 139
Signal-processing techniques for cochlear implants
Laizou, P. C. (1999) · IEEE Engineering in Medicine and Biology Magazine · doi:10.1109/51.765187 · Cited by 116
Development of visual Neuroprostheses: trends and challenges.
Fernandez, E. (2018) · Bioelectronic Medicine · doi:10.1186/s42234-018-0013-8 · Cited by 114
Self-citationMind Reading and Writing: The Future of Neurotechnology
Roelfsema, P. R., Denys, D., & Klink, P. C. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.04.001 · Cited by 107
Development of a cortical visual neuroprosthesis for the blind: the relevance of neuroplasticity
Fernández, E., Pelayo, F., Romero, S., Bongard, M., Marin, C., Alfaro, A., & Merabet, L. (2005) · Journal of Neural Engineering · doi:10.1088/1741-2560/2/4/r01 · Cited by 105
Self-citationDavis, T. S., Parker, R. A., House, P. A., Bagley, E., Wendelken, S., Normann, R. A., & Greger, B. (2012) · Journal of Neural Engineering · doi:10.1088/1741-2560/9/6/065003 · Cited by 91
High-resolution spatio-temporal mapping of visual pathways using multi-electrode arrays
Normann, R. A., Warren, D. J., Ammermüller, J., Fernández, E., & Guillory, S. G. (2001) · Vision Research · doi:10.1016/s0042-6989(00)00273-x · Cited by 90
Self-citationPixium Vision: First Clinical Results and Innovative Developments
Hornig, R., Dapper, M., Joliff, E. L., Hill, R. G., Ishaque, K., Posch, C., Benosman, R. B., Mer, Y. L., Sahel, J., & Picaud, S. A. (2016) · Springer eBooks · doi:10.1007/978-3-319-41876-6_8 · Cited by 80
Vassanelli, S., & Mahmud, M. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00438 · Cited by 78
Rao, R. P. N. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.03.008 · Cited by 77
FLORA™: phase I development of a functional vision assessment for prosthetic vision users
Geruschat, D. R., Flax, M., Tanna, N., Bianchi, M., Fisher, A., Goldschmidt, M., Fisher, L., Dagnelie, G., Deremeik, J., Smith, A., et al. (2015) · Clinical and Experimental Optometry · doi:10.1111/cxo.12242 · Cited by 75
Visual prosthesis: interfacing stimulating electrodes with retinal neurons to restore vision.
Barriga-Rivera, A., Bareket, L., Goding, J., Aregueta-Robles, U. A., & Suaning, G. J. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00620 · Cited by 73
High-Fidelity Reproduction of Spatiotemporal Visual Signals for Retinal Prosthesis
Jepson, L. H., Hottowy, P., Weiner, G. A., Dabrowski, W., Litke, A. M., & Chichilnisky, E. J. (2014) · Neuron · doi:10.1016/j.neuron.2014.04.044 · Cited by 71
Cortical visual prostheses: From microstimulation to functional percept
Najarpour Foroushani, A., Pack, C. C., & Sawan, M. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aaa904 · Cited by 62
Semantic and structural image segmentation for prosthetic vision
Sanchez-Garcia, M., Martinez-Cantin, R., & Guerrero, J. J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0227677 · Cited by 62
pulse2percept: A Python-based simulation framework for bionic vision
Beyeler, M., Boynton, G., Fine, I., & Rokem, A. (2017) · Proceedings of the 16th Python in Science Conference · doi:10.25080/shinma-7f4c6e7-00c · Cited by 56
A Probabilistic Model to Predict Clinical Phenotypic Traits from Genome Sequencing
Chen, Y., Douville, C. B., Wang, C., Niknafs, N., Yeo, G. H. T., Guthrie, V. B., Carter, H. K., Stenson, P. D., Cooper, D. N., Li, B., et al. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003825 · Cited by 55
A model of threshold for pulsatile electrical stimulation of cochlear implants
Shannon, R. V. (1989) · Hearing Research · doi:10.1016/0378-5955(89)90160-3 · Cited by 53
Introducing the Monash vision group's cortical prosthesis
Lowery, A. J. (2013) · 2013 IEEE International Conference on Image Processing · doi:10.1109/icip.2013.6738316 · Cited by 45
Visual prosthetics 2006: assessment and expectations
Dagnelie, G. (2006) · Expert Review of Medical Devices · doi:10.1586/17434440.3.3.315 · Cited by 41
Roe, A. W., Chernov, M. M., Friedman, R. M., & Chen, G. (2015) · Frontiers in Neuroanatomy · doi:10.3389/fnana.2015.00135 · Cited by 32
Simulation of visual perception and learning with a retinal prosthesis
Golden, J. R., Erickson-Davis, C., Cottaris, N. P., Parthasarathy, N., Rieke, F., Brainard, D. H., Wandell, B. A., & Chichilnisky, E. J. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/aaf270 · Cited by 29
Subtilisin Genes (SUB1-3) Presence in Microsporum canis Isolates with Human and Animal Source
Khalili, M., Bayat, M., Hazaveh, S. J. H., et al. (2019) · Research in Molecular Medicine · doi:10.18502/rmm.v6i1.3919 · Cited by 28
Apfelbaum, H. L., Pelah, A., & Eli Peli, A. (2007) · ACM Transactions on Applied Perception · doi:10.1145/1227134.1227142 · Cited by 23
A space-variant model for motion interpretation across the visual field
Chessa, M., Maiello, G., Bex, P. J., & Solari, F. (2016) · Journal of Vision · doi:10.1167/16.2.12 · Cited by 22
Solari, F., Chessa, M., & Sabatini, S. P. (2011) · Pattern Recognition Letters · doi:10.1016/j.patrec.2011.09.021 · Cited by 22
Lozano, A., Soto-Sánchez, C., Garrigós, J., Martínez, J. J., Ferrández, J. M., & Fernández, E. (2018) · International Journal of Neural Systems · doi:10.1142/s0129065718500430 · Cited by 16
Self-citationPhosphene Mapping Techniques for Visual Prostheses
Stronks, H. C., & Dagnelie, G. (2011) · Springer eBooks · doi:10.1007/978-1-4419-0754-7_19 · Cited by 14
A neuromorphic control module for real-time vergence eye movements on the iCub robot head
Gibaldi, A., Canessa, A., Chessa, M., et al. (2011) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2011.6100861 · Cited by 14
Simulating neuroprosthetic vision for emotion recognition
Bollen, C. J. M., Guclu, U., van Wezel, R. J. A., van Gerven, M. A. J., & Gucluturk, Y. (2019) · 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw.2019.8925229 · Cited by 10
On the Automatic Tuning of a Retina Model by Using a Multi-objective Optimization Genetic Algorithm
Crespo-Cano, R., Martínez-Álvarez, A., Díaz‐Tahoces, A., Cuenca-Asensi, S., Ferrández, J. M., & Fernández, E. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-18914-7_12 · Cited by 7
Self-citationDevelopment of a Cortical Visual Neuroprosthesis for the Blind: Preliminary results
Fernández, E., Soto, C., Alfaro, A., González, P., Lozano, A., Peña, S., Grima, M. D., Doblado, A. R., Alarcón, A., Rolston, J. D., et al. (2019) · Investigative Ophthalmology & Visual Science · Cited by 4
Self-citation(2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2920789 · Cited by 3
Research works printed but not linked: 12
- House W. F., 1976, Otol. Rhinol. Laryngol., 85, 1
- Mcintosh L., 2016, Advances in Neural Information Processing Systems, 29, 1369
- Bradski G., 2000, Dr. Dobb’s J. Softw. Tools, 25, 120
- Oliphant T. E., 2006, Guide to NumPy
- Cadieu C. F., 2014, PLoS Comput. Biol., 10, 1
- Marblestone A. H., 2016, Front. Comput. Neurosci., 10
- Roe1 A. W., 2015, Front. Neuroanat., 9, 135
- Lozano A., 2019, Lect. Notes Comput. Sci., 11486, 1
- Vicente J. M. Ferrandez, 2019, Proc. of the th Int. Work-Conf. the Interplay Between Natural and Artificial Computation
- Vicente J. M. Ferrandez, 2019, From Bioinspired Systems and Biomedical Applications to Machine Learning, Proc. of the th Int. Work-Conf. the Interplay Between Natural and Artificial Computation
- Romero S., 2008, Proc. of the First Int. Conf. Biomedical Electronics and Devices, 84
- Fernandez E., 2019, Invest. Ophthalmol. Vis. Sci., 60, 4021
Cited by
Self-Supervised Learning for Electroencephalography
Rafiei, M. H., Gauthier, L. V., Adeli, H., & Takabi, D. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3190448 · Cited by 317
“Machine learning techniques, such as deep artificial neural networks [13]–[15], mathematically/statistically simulate the human brain’s ability to learn, recognize, discover, estimate, and interpret complicated patterns of sensory inputs [16]–[18].”
BackgroundFernández, E., Alfaro, A., Soto-Sánchez, C., Gonzalez-Lopez, P., Lozano, A. M., Peña, S., Grima, M. D., Rodil, A., Gómez, B., Chen, X., et al. (2021) · Journal of Clinical Investigation · doi:10.1172/jci151331 · Cited by 246
Self-citationMirzaei, G., & Adeli, H. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.103293 · Cited by 211
Integrating structural control, health monitoring, and energy harvesting for smart cities
Javadinasab Hormozabad, S., Gutierrez Soto, M., & Adeli, H. (2021) · Expert Systems · doi:10.1111/exsy.12845 · Cited by 158
Nogay, H. S., & Adeli, H. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2022.104234 · Cited by 116
Metadata
- Authors
- Lozano, A., Suárez, J. S., Soto-Sánchez, C., Garrigós, J., Martínez-Alvarez, J. J., Ferrández, J. M., & Fernández, E.
- Affiliations
- Universidad Politécnica de Cartagena (ES)
- Universitat de Miguel Hernández d'Elx (ES)
- Instituto de Bioingeniería, Universidad Miguel Hernández, 03202 Alicante, Spain
- Departamento de Electrónica, Tecnología de Computadoras y Proyectos, Universidad Politécnica de Cartagena, 30202 Cartagena, Spain
- Venue
- International Journal of Neural Systems
- Publisher
- World Scientific Pub Co Pte Ltd
- Published
- 2020-07-21
- Volume
- 30
- Issue
- 09
- Pages
- 2050045
- ISSN
- 0129-0657 (ISSN-L)
- 1793-6462
- OpenAlex
- W3025652821
- Cited by
- 57
- References
- 65
- Citations by year
- 2026: 8 · 2025: 9 · 2024: 5 · 2023: 7 · 2022: 15 · 2021: 13
- FWCI
- 4.01
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Neural Systems (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Fundacion Seneca Agencia de Ciencia y Tecnologia de la Region de Murcia (20041/GERM/16, 21091/PDC/19, RTI2018-098969-B-100)
- Spanish Ministerio de Ciencia Innovacion y Universidades (PROMETEO/2019/119)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neural Engineering
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- cortical visual prosthesis, intracortical microelectrodes, visual encoding, semantic segmentation, object detection, neurostimulation devices, visual cortex
- MeSH terms
- Electrocorticography, Deep Learning, Blindness, Cerebral Cortex, Equipment Design, Humans, Image Interpretation, Computer-Assisted, Models, Theoretical, Retinal Ganglion Cells, Software Design, Visual Prosthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.