CoChat
Sign Up
Cited by 336Metamaterials and Metasurfaces ApplicationsNeural Networks and Reservoir ComputingAdvanced Memory and Neural Computing

The DOI matches its Crossref record; 74 of the 76 references it cites were found in a registry or an index.

Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible

Open access
Luo, Xuhao, Hu, Yueqiang, Ou, Xiangnian, Li, Xin, Lai, Jiajie, Liu, Na, Cheng, Xinbin, Pan, Anlian, Duan, Huigao·2022·Light: Science & Applications
DOI 10.1038/s41377-022-00844-2PMC PMC9142536
Open this paper in CoChat — free accountOpen PDF

Abstract

Replacing electrons with photons is a compelling route toward high-speed, massively parallel, and low-power artificial intelligence computing. Recently, diffractive networks composed of phase surfaces were trained to perform machine learning tasks through linear optical transformations. However, the existing architectures often comprise bulky components and, most critically, they cannot mimic the human brain for multitasking. Here, we demonstrate a multi-skilled diffractive neural network based on a metasurface device, which can perform on-chip multi-channel sensing and multitasking in the visible. The polarization multiplexing scheme of the subwavelength nanostructures is applied to construct a multi-channel classifier framework for simultaneous recognition of digital and fashionable items. The areal density of the artificial neurons can reach up to 6.25 × 10 6 mm −2 multiplied by the number of channels. The metasurface is integrated with the mature complementary metal-oxide semiconductor imaging sensor, providing a chip-scale architecture to process information directly at physical layers for energy-efficient and ultra-fast image processing in machine vision, autonomous driving, and precision medicine.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41377-022-00844-2

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Multiplexed human brain perception system and schematic of multiplexed metasurface-based diffractive neural networks (MDNN) integrated on an imaging sensor chip. a Multi-channel senses of the human brain mainly comprise sight, hearing, smell, taste, and touch, among which vision can be subdivided into object recognition, gesture recognition, character recognition and face recognition, etc. b Architecture of the MDNN. The meta-neurons of the multiple networks are trained individually to obtain multiplexed phase distributions, which are optimized by an error backpropagation algorithm running in computer. c Optical layout of polarization-dependent object classification for the MDNN concept. The input is the light carrying information about the object to be recognized, e.g., a handwritten digit or a fashion product in x- and y-polarization, respectively. The hidden layers consist of polarization-multiplexed metasurfaces acting as neurons, and subsequently converge the diffraction energy to the corresponding photoelectric detection region on the CMOS chip (i.e., the output layer of network)
Fig. 1 Multiplexed human brain perception system and schematic of multiplexed metasurface-based diffractive neural networks (MDNN) integrated on an imaging sensor chip. a Multi-channel senses of the human brain mainly comprise sight, hearing, smell, taste, and touch, among which vision can be subdivided into object recognition, gesture recognition, character recognition and face recognition, etc. b Architecture of the MDNN. The meta-neurons of the multiple networks are trained individually to obtain multiplexed phase distributions, which are optimized by an error backpropagation algorithm running in computer. c Optical layout of polarization-dependent object classification for the MDNN concept. The input is the light carrying information about the object to be recognized, e.g., a handwritten digit or a fashion product in x- and y-polarization, respectively. The hidden layers consist of polarization-multiplexed metasurfaces acting as neurons, and subsequently converge the diffraction energy to the corresponding photoelectric detection region on the CMOS chip (i.e., the output layer of network)CC BY · PubMed Central
Fig. 2 The training and design of the MDNN. a Variation of MNIST classification accuracy with respect to the number of hidden layers and classes. b Performance comparisons of MNIST and Fashion-MNIST classifiers with 10-class as an example. c Classification accuracy of MNIST dataset as a function of different percentage of detected regions. d Schematic of a single TiO2 meta-unit with a fixed height H, while tunable structure dimensions Dx and Dy. Each meta-unit acts as a neuron that has multiplexed phase profiles trained by machine learning. e, f Simulated values of the transmission coefficients (Txx, Tyy) and the phase shifts (φxx, φyy) under x- and y-polarized light, respectively. An incident wavelength of 532 nm, a nanopillar period of 400 nm, and a height of 600 nm are assumed. g Simulated output patterns and energy distribution percentages in the detection plane corresponding to a handwritten input of “3”, in the case of phase-only and amplitude-crosstalk networks, respectively. The confusion matrixes for h phase-only (amplitude is considered to be a constant value) and i amplitude-crosstalk MDNN are demonstrated, where the amplitude-crosstalk is based on the complex amplitude of the meta-unit in c. Pct. percentage. j Percentage error matrix between h and i. k Percentage of energy distribution for each digit of the network with amplitude-crosstalk. The error bars show the differences in comparison with the phase-only network
Fig. 2 The training and design of the MDNN. a Variation of MNIST classification accuracy with respect to the number of hidden layers and classes. b Performance comparisons of MNIST and Fashion-MNIST classifiers with 10-class as an example. c Classification accuracy of MNIST dataset as a function of different percentage of detected regions. d Schematic of a single TiO2 meta-unit with a fixed height H, while tunable structure dimensions Dx and Dy. Each meta-unit acts as a neuron that has multiplexed phase profiles trained by machine learning. e, f Simulated values of the transmission coefficients (Txx, Tyy) and the phase shifts (φxx, φyy) under x- and y-polarized light, respectively. An incident wavelength of 532 nm, a nanopillar period of 400 nm, and a height of 600 nm are assumed. g Simulated output patterns and energy distribution percentages in the detection plane corresponding to a handwritten input of “3”, in the case of phase-only and amplitude-crosstalk networks, respectively. The confusion matrixes for h phase-only (amplitude is considered to be a constant value) and i amplitude-crosstalk MDNN are demonstrated, where the amplitude-crosstalk is based on the complex amplitude of the meta-unit in c. Pct. percentage. j Percentage error matrix between h and i. k Percentage of energy distribution for each digit of the network with amplitude-crosstalk. The error bars show the differences in comparison with the phase-only networkCC BY · PubMed Central
Fig. 3 Vector simulations of multi-layer MDNN. a Flowcharts of scalar diffraction calculation and vector FDTD simulation. b Comparison of simulation results between scalar diffraction calculation and vector FDTD calculation for multi-channel classification. c The electric field distribution in the z-plane simulated by the handwritten input of “3” in b demonstrates that the light propagation is focused on the target region. d The output intensity in b is normalized along the x-direction distribution, and the maximum peaks are all confined to the detection region. R region. e The average energy distribution of simulated 10 groups for each of the four types of objects, all of which are randomly selected in the MNIST and Fashion-MNIST datasets, reached more than 80%
Fig. 3 Vector simulations of multi-layer MDNN. a Flowcharts of scalar diffraction calculation and vector FDTD simulation. b Comparison of simulation results between scalar diffraction calculation and vector FDTD calculation for multi-channel classification. c The electric field distribution in the z-plane simulated by the handwritten input of “3” in b demonstrates that the light propagation is focused on the target region. d The output intensity in b is normalized along the x-direction distribution, and the maximum peaks are all confined to the detection region. R region. e The average energy distribution of simulated 10 groups for each of the four types of objects, all of which are randomly selected in the MNIST and Fashion-MNIST datasets, reached more than 80%CC BY · PubMed Central
Fig. 4 Fabrication and optical verification of the dual-channel classifier MDNN. a Optical micrographs of masks formed after Al deposition and coated with a spacer layer (left), and the finally fabricated MDNN devices (right). b Top-view (left), oblique-view (middle), and false-color cross-sectional view (right) of the scanning electron microscope (SEM) images, respectively, of the MDNN. c Simulation inputs and outputs of the dual-class classification MDNN under orthogonal polarization, and their energy distribution. d The optical demonstration of the MDNN surface (top), the output field intensity detected by a camera for these cases (middle), and energy distribution of the experiment results (bottom)
Fig. 4 Fabrication and optical verification of the dual-channel classifier MDNN. a Optical micrographs of masks formed after Al deposition and coated with a spacer layer (left), and the finally fabricated MDNN devices (right). b Top-view (left), oblique-view (middle), and false-color cross-sectional view (right) of the scanning electron microscope (SEM) images, respectively, of the MDNN. c Simulation inputs and outputs of the dual-class classification MDNN under orthogonal polarization, and their energy distribution. d The optical demonstration of the MDNN surface (top), the output field intensity detected by a camera for these cases (middle), and energy distribution of the experiment results (bottom)CC BY · PubMed Central
Fig. 5 Experimental demonstration of the on-chip MDNN. a Exploded schematic diagram of the MDNN integrated with a CMOS chip. b Physical photograph of the on-chip MDNN. c Optical micrograph of the fabricated MDNN built on a CMOS imaging sensor. d The fabricated MDNN (1st and 4th columns), the output field intensity detected by the CMOS imaging sensor for these four-category classification MDNN in x- or y-polarization (2nd and 5th columns), and experimentally detected energy distribution (3rd and 6th columns). e Experimental confusion matrices for MNIST and Fashion-MNIST classification, respectively, with 80 images randomly selected from the correct set of simulations, for counting in the four classes (i.e., 20 per class). Pct. percentage
Fig. 5 Experimental demonstration of the on-chip MDNN. a Exploded schematic diagram of the MDNN integrated with a CMOS chip. b Physical photograph of the on-chip MDNN. c Optical micrograph of the fabricated MDNN built on a CMOS imaging sensor. d The fabricated MDNN (1st and 4th columns), the output field intensity detected by the CMOS imaging sensor for these four-category classification MDNN in x- or y-polarization (2nd and 5th columns), and experimentally detected energy distribution (3rd and 6th columns). e Experimental confusion matrices for MNIST and Fashion-MNIST classification, respectively, with 80 images randomly selected from the correct set of simulations, for counting in the four classes (i.e., 20 per class). Pct. percentageCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 76 references

Research works printed but not linked: 1

  1. Xiao, H., Rasul, K. & Vollgraf, R. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv 1708, 07747 (2017).

Cited by

Sorted by citations

Items 301–336 of 336

Related

See all

Metadata

Authors
Luo, X., Hu, Y., Ou, X., Li, X., Lai, J., Liu, N., Cheng, X., Pan, A., & Duan, H.
Affiliations
  • Tongji University (CN)
  • Hunan University (CN)
  • University of Stuttgart (DE)
Corresponding author
  • Hu, Yueqiang
  • Duan, Huigao
ORCID
Venue
Light: Science & Applications
Publisher
Springer Science and Business Media LLC
Published
2022-05-27
Volume
11
Issue
1
Pages
158-158
ISSN
2047-7538 (ISSN-L)
DOI
10.1038/s41377-022-00844-2
PMC
PMC9142536
OpenAlex
W3184400519
Cited by
335
References
76
Citations by year
2026: 58 · 2025: 105 · 2024: 100 · 2023: 56 · 2022: 15
FWCI
15.91
Type
journal article
Language
en
Open access
Licence
Locations
  • Light Science & Applications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
National Natural Science Foundation of China (52005175)
Topics
  • Physical Sciences › Materials Science › Electronic, Optical and Magnetic Materials › Metamaterials and Metasurfaces Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
Keywords
metasurfaces, diffractive neural networks, polarization multiplexing, on-chip image processing, optical neuromorphic computing, artificial neurons

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review