The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
High-quality hyperspectral reconstruction using a spectral prior
Open accessAbstract
We present a novel hyperspectral image reconstruction algorithm, which overcomes the long-standing tradeoff between spectral accuracy and spatial resolution in existing compressive imaging approaches. Our method consists of two steps: First, we learn nonlinear spectral representations from real-world hyperspectral datasets; for this, we build a convolutional autoencoder which allows reconstructing its own input through its encoder and decoder networks. Second, we introduce a novel optimization method, which jointly regularizes the fidelity of the learned nonlinear spectral representations and the sparsity of gradients in the spatial domain, by means of our new fidelity prior. Our technique can be applied to any existing compressive imaging architecture, and has been thoroughly tested both in simulation, and by building a prototype hyperspectral imaging system. It outperforms the state-of-the-art methods from each architecture, both in terms of spectral accuracy and spatial resolution, while its computational complexity is reduced by two orders of magnitude with respect to sparse coding techniques. Moreover, we present two additional applications of our method: hyperspectral interpolation and demosaicing. Last, we have created a new high-resolution hyperspectral dataset containing sharper images of more spectral variety than existing ones, available through our project website.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
(2010) · Cited by 14,134
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Deep learning-based classification of hyperspectral data
Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2014.2329330 · Cited by 2,679
A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration
Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.909319 · Cited by 2,064 · cited 1×
“We compare our results against three other state-of-the-art methods, representing the three different encoding architectures: TwIST [Bioucas-Dias and Figueiredo 2007], SpaRSA [Wright et al. 2009], and sparse coding [Lin et al. 2014].”
ResultStacked Convolutional Auto-Encoders for Hierarchical Feature Extraction
Masci, J., Meier, U., Cireşan, D., & Schmidhuber, J. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-21735-7_7 · Cited by 1,955
Sparse Reconstruction by Separable Approximation
Wright, S. J., Nowak, R. D., & Figueiredo, M. A. T. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2016892 · Cited by 1,854
Convolutional neural networks for large-scale remote-sensing image classification
Maggiori, E., Tarabalka, Y., Charpiat, G., & Alliez, P. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2612821 · Cited by 1,114
Single disperser design for coded aperture snapshot spectral imaging
Wagadarikar, A., John, R., Willett, R., & Brady, D. (2008) · Applied Optics · doi:10.1364/ao.47.000b44 · Cited by 1,047
Afonso, M. V., Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2076294 · Cited by 938
Single-shot compressive spectral imaging with a dual-disperser architecture
Gehm, M. E., John, R., Brady, D. J., Willett, R. M., & Schulz, T. J. (2007) · Optics Express · doi:10.1364/oe.15.014013 · Cited by 787
Afonso, M. V., Bioucas‐Dias, J. M., & Figueiredo, M. A. T. (2011) · Cited by 783
ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements
Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., & Ashok, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.55 · Cited by 768
Statistics of real-world hyperspectral images
Chakrabarti, A., & Zickler, T. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995660 · Cited by 620
Stacked Convolutional Denoising Auto-Encoders for Feature Representation
Du, B., Xiong, W., Wu, J., Zhang, L., Zhang, L., & Tao, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2536638 · Cited by 416
Zabalza, J., Ren, J., Zheng, J., Zhao, H., Qing, C., Yang, Z., Du, P., & Marshall, S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.11.044 · Cited by 389
Colorimetric and photometric properties of a 2° fundamental observer
Vos, J. J. (1978) · Color Research & Application · doi:10.1002/col.5080030309 · Cited by 379
Decomposable Nonlocal Tensor Dictionary Learning for Multispectral Image Denoising
Peng, Y., Meng, D., Xu, Z., Gao, C., Yang, Y., & Zhang, B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.377 · Cited by 359
Multiframe image estimation for coded aperture snapshot spectral imagers
Kittle, D., Choi, K., Wagadarikar, A., & Brady, D. J. (2010) · Applied Optics · doi:10.1364/ao.49.006824 · Cited by 331
Spatial-spectral encoded compressive hyperspectral imaging
Lin, X., Liu, Y., Wu, J., & Dai, Q. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661262 · Cited by 309
Hyperspectral image reconstruction by deep convolutional neural network for classification
Li, Y., Xie, W., & Li, H. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.10.019 · Cited by 264
Spectral–Spatial Classification of Hyperspectral Image Based on Deep Auto-Encoder
Ma, X., Wang, H., & Geng, J. (2016) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2016.2517204 · Cited by 258
Stacked Denoise Autoencoder Based Feature Extraction and Classification for Hyperspectral Images
Xing, C., Ma, L., & Yang, X. (2016) · Journal of Sensors · doi:10.1155/2016/3632943 · Cited by 226
A Compressive Sensing and Unmixing Scheme for Hyperspectral Data Processing
Li, C., Sun, T., Kelly, K. F., & Zhang, Y. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2167626 · Cited by 198
A Prism-Mask System for Multispectral Video Acquisition
Cao, X., Du, H., Tong, X., Dai, Q., & Lin, S. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.80 · Cited by 190
3D imaging spectroscopy for measuring hyperspectral patterns on solid objects
Kim, M. H., Harvey, T. A., Kittle, D. S., Rushmeier, H., Dorsey, J., Prum, R. O., & Brady, D. J. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185534 · Cited by 172
Self-citationDevelopment of a digital-micromirror-device-based multishot snapshot spectral imaging system
Wu, Y., Mirza, I. O., Arce, G. R., et al. (2011) · Optics Letters · doi:10.1364/ol.36.002692 · Cited by 156
Zhang, Y., Lee, K., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.06582 · Cited by 151
DeepToF: off-the-shelf real-time correction of multipath interference in time-of-flight imaging
Marco, J., Hernández, Q., Muñoz, A., Dong, Y., Jarabo, A., Kim, M. H., Tong, X., & Gutierrez, D. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130884 · Cited by 146
Self-citationExploiting Spectral-Spatial Correlation for Coded Hyperspectral Image Restoration
Fu, Y., Zheng, Y., Sato, I., & Sato, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.405 · Cited by 121
High-speed hyperspectral video acquisition with a dual-camera architecture
Wang, L., Xiong, Z., Gao, D., et al. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299128 · Cited by 111
3D object retrieval with stacked local convolutional autoencoder
Leng, B., Guo, S., Zhang, X., & Xiong, Z. (2014) · Signal Processing · doi:10.1016/j.sigpro.2014.09.005 · Cited by 98
Compact single-shot hyperspectral imaging using a prism
Baek, S.-H., Kim, I., Gutiérrez, D., & Kim, M. H. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130896 · Cited by 90
Self-citationA System for 3D Modeling Frescoed Historical Buildings with Multispectral Texture Information
Brusco, N., Capeleto, S., Fedel, M., Paviotti, A., Poletto, L., Cortelazzo, G. M., & Tondello, G. (2006) · Machine Vision and Applications · doi:10.1007/s00138-006-0026-2 · Cited by 89
The Improvement of DS Evidence Theory and Its Application in IR/MMW Target Recognition
Li, Y., Chen, J., Ye, F., & Liu, D. (2015) · Journal of Sensors · doi:10.1155/2016/1903792 · Cited by 88
Coded Hyperspectral Imaging and Blind Compressive Sensing
Rajwade, A. V., Kittle, D. S., Tsai, T.-H., Brady, D. J., & Carin, L. (2013) · SIAM Journal on Imaging Sciences · doi:10.1137/120875302 · Cited by 75
Compressive Source Separation: Theory and Methods for Hyperspectral Imaging
Golbabaee, M., Arberet, S., & Vandergheynst, P. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2281405 · Cited by 67
Compressive Source Separation: Theory and Methods for Hyperspectral Imaging
Golbabaee, M., Ieee, S. M., Arberet, S., & Pierre, V. (2012) · Cited by 61
Practical Spectral Photography
Habel, R., Kudenov, M. W., & Wimmer, M. H. (2012) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2012.03024.x · Cited by 56
Hyperspectral imaging: potential in non-destructive analysis of palimpsests
Rapantzikos, K. E., & Balas, C. J. (2005) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2005.1530131 · Cited by 51
Joint segmentation and reconstruction of hyperspectral data with compressed measurements
Zhang, Q., Plemmons, R. J., Kittle, D. S., et al. (2011) · Applied Optics · doi:10.1364/ao.50.004417 · Cited by 45
Multisampling Compressive Video Spectroscopy
Jeon, D. S., Choi, I., & Kim, M. H. (2016) · Computer Graphics Forum · doi:10.1111/cgf.12847 · Cited by 12
Self-citation3D Graphics Techniques for Capturing and Inspecting Hyperspectral Appearance
Kim, M. H. (2013) · International Symposium on Ubiquitous Virtual Reality · doi:10.1109/isuvr.2013.15 · Cited by 11
Self-citation
Cited by
Snapshot Compressive Imaging: Theory, Algorithms, and Applications
Yuan, X., Brady, D. J., & Katsaggelos, A. K. (2021) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.3023869 · Cited by 413
Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
Cai, Y., Lin, J., Hu, X., Wang, H., Yuan, X., Zhang, Y., Timofte, R., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01698 · Cited by 353
Spectral imaging with deep learning
Huang, L., Luo, R., Liu, X., & Hao, X. (2022) · Light: Science & Applications · doi:10.1038/s41377-022-00743-6 · Cited by 283
Metadata
- Authors
- Choi, I., Jeon, D. S., Nam, G., Gutierrez, D., & Kim, M. H.
- Venue
- ACM Transactions on Graphics
- Published
- 2017-11-20
- Volume
- 36
- Issue
- 6
- Pages
- 1-13
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2770113520
- Cited by
- 320
- References
- 47
- FWCI
- 7.98
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.