The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
A versatile deep learning architecture for classification and label-free prediction of hyperspectral images
Open accessAbstract
Hyperspectral imaging is a technique that provides rich chemical or compositional information not regularly available to traditional imaging modalities such as intensity imaging or color imaging based on the reflection, transmission, or emission of light. Analysis of hyperspectral imaging often relies on machine learning methods to extract information. Here, we present a new flexible architecture, the U-within-U-Net, that can perform classification, segmentation, and prediction of orthogonal imaging modalities on a variety of hyperspectral imaging techniques. Specifically, we demonstrate feature segmentation and classification on the Indian Pines hyperspectral dataset and simultaneous location prediction of multiple drugs in mass spectrometry imaging of rat liver tissue. We further demonstrate label-free fluorescence image prediction from hyperspectral stimulated Raman scattering microscopy images. The applicability of the U-within-U-Net architecture on diverse datasets with widely varying input and output dimensions and data sources suggest that it has great potential in advancing the use of hyperspectral imaging across many different application areas ranging from remote sensing, to medical imaging, to microscopy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, we remark that our unsupervised autoencoder models do not utilize any ground truth information during training, in contrast with previous supervised machine learning approaches for Raman cell imaging and characterization where specific target concentrations are used as labels to train the model (e.g., using separately acquired fluorescence images) (90–93).”
From Hyperspectral unmixing for Raman spectroscopy via physics-constrained autoencoders (2024) · cites this paper as [90]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“For additional comparisons with another modern architecture used for image reconstructions, a U-Net utilizing ResNet Blocks 36,67 was also trained to predict the organelles (Extended Data Figs.”
MethodU-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
FSIM: A Feature Similarity Index for Image Quality Assessment
Zhang, L., Lei Zhang, A., Mou, X., & Zhang, D. D. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2109730 · Cited by 5,220
Chen, Y., Jiang, H., Li, C., Jia, X., & Ghamisi, P. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2584107 · Cited by 3,064
HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image Classification
Roy, S. K., Krishna, G., Dubey, S. R., & Chaudhuri, B. B. (2019) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2019.2918719 · Cited by 2,010
Deep Learning for Hyperspectral Image Classification: An Overview
Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2907932 · Cited by 2,007
U-Net: deep learning for cell counting, detection, and morphometry
Falk, T., Mai, D., Bensch, R., Çiçek, Ö., Abdulkadir, A., Marrakchi, Y., Böhm, A., Deubner, J., Jäckel, Z., Seiwald, K., et al. (2018) · Nature Methods · doi:10.1038/s41592-018-0261-2 · Cited by 1,920 · cited 1×
“…imaging, to microscopy. with image segmentation, the U-Net has seen use in a variety of applications including segmentation, label-free prediction and denoising 9,32–34 ; however, most works that utilize the U-Net in this way are not concerned with images that contain multiple spectral channels.”
Method
Research works printed but not linked: 1
- Cheng, J.-X. & Xie, X. S. Coherent Raman Scattering Microscopy (CRC Press, 2013).
Cited by
Sorted by citations
Items 101–152 of 152
HSIS-Net: Hyperspectral Image Segmentation Using Multi-view Active Learning Based FCSN
(2023) · International journal of intelligent engineering and systems · doi:10.22266/ijies2023.0430.02 · Cited by 4
Wei, H. L., & Smith, J. P. (2024) · Molecular Pharmaceutics · doi:10.1021/acs.molpharmaceut.4c00540 · Cited by 4
Qi, C., Li, K., Zhou, M. Q., Zhang, C., Zheng, X., Chen, Q., & Hu, T. (2024) · Journal of Hazardous Materials Advances · doi:10.1016/j.hazadv.2024.100489 · Cited by 4
Kopriva, I., Sitnik, D., Dion‐Bertrand, L., Periša, M. M., Pačić, A., Hadžija, M., & Hadžija, M. P. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110841 · Cited by 3
Virtual fluorescence labeling of mitochondria in phase images using a network with unpaired datasets
Xiong, Z., Wang, W., Ma, Y., Zhou, J., Feng, W., Ali, N., An, S., & Gao, P. (2025) · Photonics Research · doi:10.1364/prj.575781 · Cited by 3
Spatial-Spectral Deep Learning for Prostate Cancer Tissue Classification in Infrared Spectroscopy
O'Leary, L., Ferguson, D., Hart, C. A., Brown, M. D., Oliveira, P., Clarke, N. W., Sachdeva, A., Gardner, P., & Yin, H. (2026) · Analytical Chemistry · doi:10.1021/acs.analchem.5c04765 · Cited by 3
Shao, S., Yan, L., Li, J., Zhang, Y., Zhang, J., Kim, H. W., & Kim, S. S. (2025) · ACS Sensors · doi:10.1021/acssensors.4c03085 · Cited by 5
Si, Y., & He, S. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3574423 · Cited by 3
Wang, B., Qinglin, Y., Wang, Z., Weitao, L., Wang, Y., Liu, H., Shuangxi, G., Lingling, Z., & Dong, L. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3581338 · Cited by 3
Li, S., Chen, B., Wang, N., Shi, Y., Zhang, G., & Liu, J. (2024) · Electronics · doi:10.3390/electronics13132540 · Cited by 3
Multilayer spatial-spectral segmentation network for detecting AFB1
Zhao, Y., Liu, H., Zhai, X., Zhang, R., Shi, W., Zhao, L., & Han, Z. (2024) · Journal of Food Composition and Analysis · doi:10.1016/j.jfca.2024.106790 · Cited by 3
Yang, X., Li, Z., Lei, L., Shi, X., Zhang, D., Zhou, F., Li, W., Xu, T., Liu, X., Wang, S., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/67256 · Cited by 3
“Hyperspectral imaging (HSI) consists of narrower bands, is a technique that captures and analyzes the details of each band in a region, can therefore detect substances that are visually indistinguishable to humans, and has a wide range of applications in many fields [13].”
Label-free metabolic fingerprinting of motile mammalian spermatozoa with subcellular resolution
Kusari, F., Backová, L., Pánek, D., Benda, A., & Trachtulec, Z. (2025) · BMC Biology · doi:10.1186/s12915-025-02167-1 · Cited by 2
Ali, M. S. M., Islam, M. S. B., Majid, M. E., Kashem, S. B. A., Khandakar, A. A., & Chowdhury, M. E. H. (2025) · Remote Sensing Applications Society and Environment · doi:10.1016/j.rsase.2025.101779 · Cited by 2
Self-calibrated three-dimensional digital image correlation based on deep learning
Dong, S., Chen, Y., Kuang, Z., Kang, X., & Ma, J. (2025) · Measurement Science and Technology · doi:10.1088/1361-6501/ae2cb9 · Cited by 2
Chemical Imaging by Stimulated Raman Scattering Microscopy
Lang, X., & Min, W. (2021) · ACS symposium series · doi:10.1021/bk-2021-1398.ch011 · Cited by 2
Bakiya, A., Anitha, A., Valliammai, M., Kumar, K. L., Kumar, S. T., & Ram, K. J. S. (2024) · International Conference on Electronics and Sustainable Communication Systems (ICESC) · doi:10.1109/icesc60852.2024.10689911 · Cited by 2
“The conventional technique for identifying API in powder, High-Performance Liquid Chromatography (HPLC), is a costly, invasive, and time-consuming procedure, which makes it unsuitable for continuous monitoring activities [4-5].”
MethodDeep neural network-based feature selection with local false discovery rate estimation
Cao, Z., Sun, X., & Fu, Y. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05944-7 · Cited by 2
Ma, Y., Mizuguchi, T., Spratt, S. J., & Ozeki, Y. (2025) · Advanced Chemical Microscopy for Life Science and Translational Medicine · doi:10.1117/12.3041987 · Cited by 1
Deep Learning‐Assisted Coherent Raman Scattering Microscopy
Liu, J., Liu, R., Zhang, X., Chen, X. G., & Yue, S. (2025) · Advanced Intelligent Discovery · doi:10.1002/aidi.202500072 · Cited by 1
Ngan, H., Zhang, J., Kwan, K. K., Cheu, J. W., Li, Z., Guo, Y., Yang, X., Wong, C. C., Yan, H. Y., & Cai, Z. (2025) · Analytical Chemistry · doi:10.1021/acs.analchem.5c05874 · Cited by 2
Artificial Intelligence for Predicting Pregnancy Risk Factors: A Literature Review
Lakhdadi, K., Boujraine, K., & Satori, H. (2025) · International Conference on Embedded Systems and Artificial Intelligence (ESAI) · doi:10.1109/esai67033.2025.11438364 · Cited by 1
“In maternal healthcare, this technology AI enables sophisticated analysis of medical data, offering new approaches for predictive modeling and early risk detection [27], [28], [29].”
Coherent nonlinear vibrational microscopy
Luo, K., Hu, D., Li, J., Yu, Y., Xue, K., Ji, Y., Yang, F., Ao, J., & Ji, M. (2026) · Applied Physics Reviews · doi:10.1063/5.0313530 · Cited by 1
Deep unfolding blind source unmixing for multicolor fluorescence imaging
Zhu, S., Li, W., Qin, Y., Cao, X., & Deng, Y. (2026) · Biomedical Optics Express · doi:10.1364/boe.605821 · Cited by 1
“The UwU-Net model [24], an improvement based on the U-Net architecture, has demonstrated remarkable unmixing capability across multiple imaging modalities.”
Mugdha, A. C., & Wilson, J. W. (2023) · Advanced Chemical Microscopy for Life Science and Translational Medicine · doi:10.1117/12.2668708 · Cited by 1
“A notable contribution to chemical imaging is the U-within-a-U network developed by Dan Fu’s lab 5 , which performed well on a variety of modalities ranging from hyperspectral remote sensing to mass-spectroscopy and coherent Raman microscopy, in the context of supervised training.”
BackgroundCoherent Raman Microscopy: From Instrumentation To Applications
Fu, D. (2023) · Journal of Visualized Experiments · doi:10.3791/64882 · Cited by 1
Self-citationChen, T., Zang, Y., Zhang, J., & Zhang, D. (2023) · SPIE Medical Imaging: Image Perception, Observer Performance, and Technology Assessment · doi:10.1117/12.2654298 · Cited by 1
“We used the SRS imaging to predict fluorescence of individual organelles, a challenging task of cross-modality image prediction because organelles are tiny and difficult to have the boundary with cytosol clearly identified 22 .”
MethodBose, R., Roy, S., & Sutradhar, S. (2023) · Smart Agricultural Technology · doi:10.1016/j.atech.2023.100334 · Cited by 1
Ao, J., & Ji, M. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-95223-1.00004-7 · Cited by 1
Si, Y., Li, S., Wang, X., & He, S. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3540121 · Cited by 1
“…it provides detailed material composition information, traditional HSI has been widely used in agriculture (crop disease surveillance) [10], [11], [12], [13], remote sensing (land-cover classification) [14], [15], [16], [17], [18], [19], medicine (tumor detection) [20], [21], [22], [23], [24],…”
Research on a Burn Severity Detection Method Based on Hyperspectral Imaging
Wang, S., Gu, M., Zhang, M., & Tan, X. (2025) · Sensors · doi:10.3390/s25051330 · Cited by 1
“Many of the latest deep learning techniques have been borrowed from other fields and successfully applied in medical imaging, yielding excellent results [33].”
Zhanghao, K., Li, M., Chen, X., Liu, W., Wang, Y., Wu, Z., Shan, C., Wu, J., Zhang, Y., Xi, P., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.12.13.472520 · Cited by 1
Liu, W., Liu, P., Wu, X., & Du, Y. (2021) · International Conference on Public Health and Data Science (ICPHDS) · doi:10.1109/icphds53608.2021.00025
Research on Visual Diagnosis of Citrus Diseased Leaves Based on Image Processing Technology
Niu, H., Shu, W., Lei, D., Zhang, S., & Chen, L. (2023) · International Conference on Electronic Information Technology and Smart Agriculture (ICEITSA) · doi:10.1145/3641343.3641455
Byler, N., Forland, B. M., & McKay, M. (2024) · Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery · doi:10.1117/12.3012800
“There is much research focusing on designing and implementing state of the art deep learning architectures for HSI data, including 3, 4, 5, 6, 7.”
BackgroundImage Classification of Hyperspectral Images using Machine Learning
Chaudhari, A. K., Zanke, T., Mulgir, S., Wath, S., & Jagtap, S. (2024) · International Conference on Electronics, Computing, Communication and Control Technology (ICECCC) · doi:10.1109/iceccc61767.2024.10593989
Zhang, Y., Zhao, Q., Li, Y., & Gong, X. (2024) · Infrared Physics & Technology · doi:10.1016/j.infrared.2024.105691
Singh, R. B., & Datta, A. S. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-93697-5_27
Coherent Raman scattering microscopy for cancer diagnosis
Ji, M. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-22253-5.00012-1
Lin, L., Bi, X., Colazo, A., Wang, H., & Wei, L. (2026) · Chemical & Biomedical Imaging · doi:10.1021/cbmi.6c00011
Chogan, H., Bagheri, R., & Parastar, H. (2025) · ACS Food Science & Technology · doi:10.1021/acsfoodscitech.5c00640
Tianyu, 秦. Q., Weijian, 钱. Q., Rui, 张. Z., Faxian, 吴. W., Junjiang, 刘. L., Ying, 林. L., Zhengkai, 吴. W., Gang, 黎. L., Yanling, 薛. X., Tiqiao, 肖. X., et al. (2025) · Laser & Optoelectronics Progress · doi:10.3788/lop251977
Revealing real-world hidden 3D atmospheric turbulence from imaging
An, Y., Jiang, X., Li, T., & Bai, X. (2025) · Newton · doi:10.1016/j.newton.2025.100364
Wang, H., Yuan, C., Zhang, H., Wang, X., Jia, H., Fu, F., Tang, X., Cui, J., Ji, J., Shi, S., et al. (2026) · Engineering · doi:10.1016/j.eng.2026.02.018
Hyperspectral imaging in oral oncology: a scoping review
Kankawale, S., Jamkhande, A., & Oka, G. (2026) · Frontiers in Oral Health · doi:10.3389/froh.2026.1857460
“This reinforces the association between light-absorption patterns and the presence of OSCC (8, 35).”
PLANCK: super-multiplex optical imaging without labelingPreprint
Liu, X., Li, X., Xu, L., Wei, M., Niaz, A., He, Y., & Min, W. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.07.02.736216
Organelle Prediction from Stimulated Raman Images by Deep Learning with a Large Dataset
Ma, Y., Morikura, T., Fukuta, S., Funahashi, A., & Ozeki, Y. (2026) · doi:10.1364/cleo_at.2026.ath2b.1
Hu, X., Xu, J., Hu, X., Deng, W., Xu, K., Jiang, L., Wang, Y., Su, S., Hu, S., & Xiang, J. H. (2026) · Spectrochimica Acta Part A Molecular and Biomolecular Spectroscopy · doi:10.1016/j.saa.2026.128493
An, Y., Jiang, Y., & Bai, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37174-4_33
Lan, Q., Zhuang, G., Wen, C., 羅世歡, Liu, H., Hu, M., & Zhuang, Z. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111560
Meeran, A. B. M., Chowdhury, M. E. H., Naznine, M., Ali, S. H. M., Sankaran, R., & Pedersen, S. (2026) · Diagnostics · doi:10.3390/diagnostics16203338
EcSMNet: edge-structured sparse mask modeling for knowledge-efficient hyperspectral reconstruction
Chen, M., Li, J., Ma, S., Gong, M., Chipeng, 曹. C., & Jiao, L. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.117229
Metadata
- Authors
- Manifold, B., Men, S., Hu, R., & Fu, D.
- Affiliations
- University of Washington (US)
- Corresponding author
- Manifold, Bryce
- Fu, Dan
- Venue
- Nature Machine Intelligence
- Published
- 2021-03-11
- Volume
- 3
- Issue
- 4
- Pages
- 306-315
- ISSN
- 2522-5839 (ISSN-L)
- PMC
- PMC8528004
- OpenAlex
- W3134761005
- Cited by
- 152
- References
- 70
- Citations by year
- 2026: 16 · 2025: 42 · 2024: 36 · 2023: 30 · 2022: 20 · 2021: 8
- FWCI
- 10.55
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Beckman Young Investigator Award
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Remote-Sensing Image Classification
- Physical Sciences › Chemistry › Analytical Chemistry › Spectroscopy and Chemometric Analyses
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- hyperspectral imaging, U-Net architecture, image classification, image segmentation, mass spectrometry imaging, stimulated Raman scattering microscopy, Indian Pines dataset, rat liver tissue, remote sensing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
