The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.
SIFT Flow: Dense Correspondence across Scenes and Its Applications
Open accessAbstract
While image alignment has been studied in different areas of computer vision for decades, aligning images depicting different scenes remains a challenging problem. Analogous to optical flow, where an image is aligned to its temporally adjacent frame, we propose SIFT flow, a method to align an image to its nearest neighbors in a large image corpus containing a variety of scenes. The SIFT flow algorithm consists of matching densely sampled, pixelwise SIFT features between two images while preserving spatial discontinuities. The SIFT features allow robust matching across different scene/object appearances, whereas the discontinuity-preserving spatial model allows matching of objects located at different parts of the scene. Experiments show that the proposed approach robustly aligns complex scene pairs containing significant spatial differences. Based on SIFT flow, we propose an alignment-based large database framework for image analysis and synthesis, where image information is transferred from the nearest neighbors to a query image according to the dense scene correspondence. This framework is demonstrated through concrete applications such as motion field prediction from a single image, motion synthesis via object transfer, satellite image registration, and face recognition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
A Combined Corner and Edge Detector
Harris, C., & Stephens, M. (1988) · Procedings of the Alvey Vision Conference 1988 · doi:10.5244/c.2.23 · Cited by 12,901
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
(1981) · Cited by 11,963
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,703
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Fast approximate energy minimization via graph cuts
Boykov, Y., Veksler, O., & Zabih, R. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.969114 · Cited by 6,988
Cited by
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Lin, G., Milan, A., Shen, C., & Reid, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.549 · Cited by 3,287
“[32] RGB-D − 10.”
VoxelMorph: A Learning Framework for Deformable Medical Image Registration
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J., & Dalca, A. V. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2897538 · Cited by 2,174
Deeper Depth Prediction with Fully Convolutional Residual Networks
Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., & Navab, N. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.32 · Cited by 1,982
SUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943
LoFTR: Detector-Free Local Feature Matching with Transformers
Sun, J., Shen, Z., Wang, Y., Bao, H., & Zhou, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00881 · Cited by 1,719
Kirillov, A., He, K., Girshick, R., Rother, C., & Dollár, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00963 · Cited by 1,659
Deformable medical image registration: a survey
Sotiras, A., Davatzikos, C., & Paragios, N. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2265603 · Cited by 1,587
“[430] used the decomposed model [428] along with loopy BP tomatch SIFT-descriptors along the flow vectors.”
MethodThe Mapillary Vistas Dataset for Semantic Understanding of Street Scenes
Neuhold, G., Ollmann, T., Bulo, S. R., & Kontschieder, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.534 · Cited by 1,434
Generative Visual Manipulation on the Natural Image Manifold
Zhu, J.-Y., Krähenbühl, P., Shechtman, E., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_36 · Cited by 1,171
Cross-scene crowd counting via deep convolutional neural networks
Zhang, C., Li, H., Wang, X., & Yang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298684 · Cited by 1,121
Unsupervised Salience Learning for Person Re-identification
Zhao, R., Ouyang, W., & Wang, X. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.460 · Cited by 1,025
DeepFlow: Large Displacement Optical Flow with Deep Matching
Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.175 · Cited by 997
Fan, D.-P., Ji, G.-P., Sun, G., Cheng, M.-M., Shen, J., & Shao, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00285 · Cited by 896
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.348 · Cited by 862
Fan, D.-P., Ji, G.-P., Cheng, M.-M., & Shao, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3085766 · Cited by 779
Deep learning in medical image registration: a survey
Haskins, G., Kruger, U., & Yan, P. (2020) · Machine Vision and Applications · doi:10.1007/s00138-020-01060-x · Cited by 704
Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks
Fan, D.-P., Lin, Z., Zhang, Z., Zhu, M., & Cheng, M.-M. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2996406 · Cited by 702
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Sariyanidi, E., Gunes, H., & Cavallaro, A. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366127 · Cited by 685
Lung Segmentation in Chest Radiographs Using Anatomical Atlases With Nonrigid Registration
Candemir, S., Jaeger, S., Palaniappan, K., Musco, J. P., Singh, R. K., Xue, Z., Karargyris, A., Antani, S., Thoma, G., & McDonald, C. J. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2290491 · Cited by 674
MUlti-Store Tracker (MUSTer): A cognitive psychology inspired approach to object tracking
Hong, Z., Chen, Z., Wang, C., Mei, X., Prokhorov, D., & Tao, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298675 · Cited by 644
An Unsupervised Learning Model for Deformable Medical Image Registration
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Dalca, A. V., & Guttag, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00964 · Cited by 599
Robust Point Matching via Vector Field Consensus
Ma, J., Zhao, J., Tian, J., Yuille, A. L., & Tu, Z. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2307478 · Cited by 566
Part-Aligned Bilinear Representations for Person Re-identification
Suh, Y., Wang, J., Tang, S., Mei, T., & Lee, K. M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_25 · Cited by 557
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics
Gong, Y., Ke, Q., Isard, M., & Lazebnik, S. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0658-4 · Cited by 540
Semi Supervised Semantic Segmentation Using Generative Adversarial Network
Souly, N., Spampinato, C., & Shah, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.606 · Cited by 513
Depth Transfer: Depth Extraction from Video Using Non-Parametric Sampling
Karsch, K., Liu, C., & Kang, S. B. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2316835 · Cited by 453
Self-citationConvolutional Neural Network Architecture for Geometric Matching
Rocco, I., Arandjelovic, R., & Sivic, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.12 · Cited by 496
Multi-focus image fusion with dense SIFT
Liu, Y., Liu, S., & Wang, Z. (2015) · Information Fusion · doi:10.1016/j.inffus.2014.05.004 · Cited by 493
Visual attribute transfer through deep image analogy
Liao, J., Yao, Y., Yuan, L., Hua, G., & Kang, S. B. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073683 · Cited by 487
Burst photography for high dynamic range and low-light imaging on mobile cameras
Hasinoff, S. W., Sharlet, D., Geiss, R., Adams, A., Barron, J. T., Kainz, F., Chen, J., & Levoy, M. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980254 · Cited by 483
Leveraging stereopsis for saliency analysis
Niu, Y., Geng, Y., Li, X., & Liu, F. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247708 · Cited by 479
An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders
Walker, J., Doersch, C., Gupta, A., & Hebert, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_51 · Cited by 449
Learning Correspondence From the Cycle-Consistency of Time
Wang, X., Jabri, A., & Efros, A. A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00267 · Cited by 445
Optical flow modeling and computation: A survey
Fortun, D., Bouthemy, P., & Kervrann, C. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.02.008 · Cited by 431
Context Contrasted Feature and Gated Multi-scale Aggregation for Scene Segmentation
Ding, H., Jiang, X., Shuai, B., Liu, A. Q., & Wang, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00254 · Cited by 394
Rethinking Semantic Segmentation: A Prototype View
Zhou, T., Wang, W., Konukoglu, E., & Van Goo, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00261 · Cited by 376
Nonparametric Scene Parsing via Label Transfer
Liu, C., Yuen, J., & Torralba, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.131 · Cited by 368
Self-citationUnsupervised Joint Object Discovery and Segmentation in Internet Images
Rubinstein, M., Joulin, A., Kopf, J., & Liu, C. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.253 · Cited by 355
Self-citationFast Optical Flow Using Dense Inverse Search
Kroeger, T., Timofte, R., Dai, D., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_29 · Cited by 351
Learning Dense Correspondence via 3D-Guided Cycle Consistency
Zhou, T., Krahenbuhl, P., Aubry, M., Huang, Q., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.20 · Cited by 350
A vision-based motion capture and recognition framework for behavior-based safety management
Han, S., & Lee, S. (2013) · Automation in Construction · doi:10.1016/j.autcon.2013.05.001 · Cited by 347
Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
Sultana, F., Sufian, A., & Dutta, P. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106062 · Cited by 339
Recycle-GAN: Unsupervised Video Retargeting
Bansal, A., Ma, S., Ramanan, D., & Sheikh, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_8 · Cited by 328
Dual Motion GAN for Future-Flow Embedded Video Prediction
Liang, X., Lee, L., Dai, W., & Xing, E. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.194 · Cited by 328
Rajaraman, S., Candemir, S., Kim, I., Thoma, G., & Antani, S. (2018) · Applied Sciences · doi:10.3390/app8101715 · Cited by 318
Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals
Song, M., Lim, S., & Kim, W. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3049869 · Cited by 299
Meta-Analysis of the First Facial Expression Recognition Challenge
Valstar, M. F., Mehu, M., Jiang, B., Pantic, M., & Scherer, K. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2200675 · Cited by 297
Deep exemplar-based colorization
He, M., Chen, D., Liao, J., Sander, P. V., & Yuan, L. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201365 · Cited by 293
Parallax-Tolerant Image Stitching Based on Robust Elastic Warping
Li, J., Wang, Z., Lai, S., Zhai, Y., & Zhang, M. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2777461 · Cited by 291
Large-Scale Video Summarization Using Web-Image Priors
Khosla, A., Hamid, R., Lin, C.-J., & Sundaresan, N. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.348 · Cited by 283
COTR: Correspondence Transformer for Matching Across Images
Jiang, W., Trulls, E., Hosang, J., Tagliasacchi, A., & Yi, K. M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00615 · Cited by 282
Ma, J., Qiu, W., Zhao, J., Ma, Y., Yuille, A. L., & Tu, Z. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2388434 · Cited by 280
Paper Doll Parsing: Retrieving Similar Styles to Parse Clothing Items
Yamaguchi, K., Kiapour, M. H., & Berg, T. L. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.437 · Cited by 275
Boundary-Aware Feature Propagation for Scene Segmentation
Ding, H., Jiang, X., Liu, A. Q., Thalmann, N. M., & Wang, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00692 · Cited by 271
Depth Extraction from Video Using Non-parametric Sampling
Karsch, K., Liu, C., & Kang, S. B. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_56 · Cited by 267
Self-citationDeepMatching: Hierarchical Deformable Dense Matching
Revaud, J., Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0908-3 · Cited by 267
A Survey of Fingerprint-Based Outdoor Localization
Vo, Q. D., & De, P. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2448632 · Cited by 259
Image colorization using similar images
Gupta, R. K., Chia, A. Y.-S., Rajan, D., Ng, E. S., & Zhiyong, H. (2012) · Proceedings of the 20th ACM international conference on Multimedia · doi:10.1145/2393347.2393402 · Cited by 256
StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2
Skorokhodov, I., Tulyakov, S., & Elhoseiny, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00361 · Cited by 199
Robust Object Co-Segmentation Using Background Prior
Han, J., Quan, R., Zhang, D., & Nie, F. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2781424 · Cited by 169
Deep learning approach to describe and classify fungi microscopic images
Zieliński, B., Sroka-Oleksiak, A., Rymarczyk, D., Piekarczyk, A., & Brzychczy-Włoch, M. (2020) · PLOS ONE · doi:10.1371/journal.pone.0234806 · Cited by 126
Yu, Z., Tan, E.-L., Ni, D., Qin, J., Chen, S., Li, S., Lei, B., & Wang, T. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2705031 · Cited by 109
Image Segmentation with Cascaded Hierarchical Models and Logistic Disjunctive Normal Networks
Seyedhosseini, M., Sajjadi, M., & Tasdizen, T. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.269 · Cited by 95
Multi-modal Unsupervised Feature Learning for RGB-D Scene Labeling
Wang, A., Lu, J., Wang, G., Cai, J., & Cham, T.-J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_30 · Cited by 50
Huang, S. H., Pan, J., Mulcaire, G., & Abbeel, P. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353475 · Cited by 39
LiveObj: Object Semantics-based Viewport Prediction for Live Mobile Virtual Reality Streaming
Feng, X., Bao, Z., & Wei, S. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3067686 · Cited by 34
Optical Flow Estimation with Deep Learning, a Survey on Recent Advances
Savian, S., Elahi, M., & Tillo, T. (2020) · Unsupervised and Semi-Supervised Learning · doi:10.1007/978-3-030-32583-1_12 · Cited by 29
Metadata
- Authors
- Liu, C., Yuen, J., & Torralba, A.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2011-05
- Volume
- 33
- Issue
- 5
- Pages
- 978-994
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2090518410
- Cited by
- 1,582
- References
- 78
- FWCI
- 54.18
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.