The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
A versatile deep learning architecture for classification and label-free prediction of hyperspectral images
Open accessAbstract
Hyperspectral imaging is a technique that provides rich chemical or compositional information not regularly available to traditional imaging modalities such as intensity imaging or color imaging based on the reflection, transmission, or emission of light. Analysis of hyperspectral imaging often relies on machine learning methods to extract information. Here, we present a new flexible architecture, the U-within-U-Net, that can perform classification, segmentation, and prediction of orthogonal imaging modalities on a variety of hyperspectral imaging techniques. Specifically, we demonstrate feature segmentation and classification on the Indian Pines hyperspectral dataset and simultaneous location prediction of multiple drugs in mass spectrometry imaging of rat liver tissue. We further demonstrate label-free fluorescence image prediction from hyperspectral stimulated Raman scattering microscopy images. The applicability of the U-within-U-Net architecture on diverse datasets with widely varying input and output dimensions and data sources suggest that it has great potential in advancing the use of hyperspectral imaging across many different application areas ranging from remote sensing, to medical imaging, to microscopy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, we remark that our unsupervised autoencoder models do not utilize any ground truth information during training, in contrast with previous supervised machine learning approaches for Raman cell imaging and characterization where specific target concentrations are used as labels to train the model (e.g., using separately acquired fluorescence images) (90–93).”
From Hyperspectral unmixing for Raman spectroscopy via physics-constrained autoencoders (2024) · cites this paper as [90]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“For additional comparisons with another modern architecture used for image reconstructions, a U-Net utilizing ResNet Blocks 36,67 was also trained to predict the organelles (Extended Data Figs.”
MethodU-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
FSIM: A Feature Similarity Index for Image Quality Assessment
Zhang, L., Lei Zhang, A., Mou, X., & Zhang, D. D. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2109730 · Cited by 5,220
Chen, Y., Jiang, H., Li, C., Jia, X., & Ghamisi, P. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2584107 · Cited by 3,064
HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image Classification
Roy, S. K., Krishna, G., Dubey, S. R., & Chaudhuri, B. B. (2019) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2019.2918719 · Cited by 2,010
Deep Learning for Hyperspectral Image Classification: An Overview
Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2907932 · Cited by 2,007
U-Net: deep learning for cell counting, detection, and morphometry
Falk, T., Mai, D., Bensch, R., Çiçek, Ö., Abdulkadir, A., Marrakchi, Y., Böhm, A., Deubner, J., Jäckel, Z., Seiwald, K., et al. (2018) · Nature Methods · doi:10.1038/s41592-018-0261-2 · Cited by 1,920 · cited 1×
“…imaging, to microscopy. with image segmentation, the U-Net has seen use in a variety of applications including segmentation, label-free prediction and denoising 9,32–34 ; however, most works that utilize the U-Net in this way are not concerned with images that contain multiple spectral channels.”
Method
Research works printed but not linked: 1
- Cheng, J.-X. & Xie, X. S. Coherent Raman Scattering Microscopy (CRC Press, 2013).
Cited by
Sorted by citations
Small molecule metabolites: discovery of biomarkers and therapeutic targets
Qiu, S., Cai, Y., Yao, H., Lin, C., Xie, Y., Tang, S., & Zhang, A. (2023) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-023-01399-3 · Cited by 993
Rapid single-particle chemical imaging of nanoplastics by SRS microscopy
Qian, N., Gao, X., Lang, X., Deng, H., Bratu, T. M., Chen, Q., Stapleton, P., Yan, B., & Min, W. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2300582121 · Cited by 467
Hyperspectral imaging and its applications: A review
Bhargava, A., Sachdeva, A., Sharma, K., Alsharif, M. H., Uthansakul, P., & Uthansakul, M. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e33208 · Cited by 363
HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model
Wang, D., Hu, M., Jin, Y., Miao, Y., Yang, J., Xu, Y., Qin, X., Ma, J., Sun, L., Li, C., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3557581 · Cited by 290
“This results in nearly continuous spectral curves that provide detailed surface information, enabling the distinction of subtle spectral differences between substances for precise land cover interpretation [4].”
Deep learning in spectral analysis: Modeling and imaging
Liu, X., An, H., Cai, W., & Shao, X. (2024) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2024.117612 · Cited by 146
Metadata
- Authors
- Manifold, B., Men, S., Hu, R., & Fu, D.
- Affiliations
- University of Washington (US)
- Corresponding author
- Manifold, Bryce
- Fu, Dan
- Venue
- Nature Machine Intelligence
- Published
- 2021-03-11
- Volume
- 3
- Issue
- 4
- Pages
- 306-315
- ISSN
- 2522-5839 (ISSN-L)
- PMC
- PMC8528004
- OpenAlex
- W3134761005
- Cited by
- 152
- References
- 70
- Citations by year
- 2026: 16 · 2025: 42 · 2024: 36 · 2023: 30 · 2022: 20 · 2021: 8
- FWCI
- 10.55
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Beckman Young Investigator Award
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Remote-Sensing Image Classification
- Physical Sciences › Chemistry › Analytical Chemistry › Spectroscopy and Chemometric Analyses
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- hyperspectral imaging, U-Net architecture, image classification, image segmentation, mass spectrometry imaging, stimulated Raman scattering microscopy, Indian Pines dataset, rat liver tissue, remote sensing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
