The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.
SIFT Flow: Dense Correspondence across Scenes and Its Applications
Open accessAbstract
While image alignment has been studied in different areas of computer vision for decades, aligning images depicting different scenes remains a challenging problem. Analogous to optical flow, where an image is aligned to its temporally adjacent frame, we propose SIFT flow, a method to align an image to its nearest neighbors in a large image corpus containing a variety of scenes. The SIFT flow algorithm consists of matching densely sampled, pixelwise SIFT features between two images while preserving spatial discontinuities. The SIFT features allow robust matching across different scene/object appearances, whereas the discontinuity-preserving spatial model allows matching of objects located at different parts of the scene. Experiments show that the proposed approach robustly aligns complex scene pairs containing significant spatial differences. Based on SIFT flow, we propose an alignment-based large database framework for image analysis and synthesis, where image information is transferred from the nearest neighbors to a query image according to the dense scene correspondence. This framework is demonstrated through concrete applications such as motion field prediction from a single image, motion synthesis via object transfer, satellite image registration, and face recognition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
A Combined Corner and Edge Detector
Harris, C., & Stephens, M. (1988) · Procedings of the Alvey Vision Conference 1988 · doi:10.5244/c.2.23 · Cited by 12,901
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
(1981) · Cited by 11,963
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,703
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Fast approximate energy minimization via graph cuts
Boykov, Y., Veksler, O., & Zabih, R. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.969114 · Cited by 6,988
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740
Video Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367
Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope
Oliva, A., & Torralba, A. (2001) · International Journal of Computer Vision · doi:10.1023/a:1011139631724 · Cited by 6,319
Self-citationShape matching and object recognition using shape contexts
Belongie, S., Malik, J., & Puzicha, J. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.993558 · Cited by 6,306
Swain, M. J., & Ballard, D. H. (1991) · International Journal of Computer Vision · doi:10.1007/bf00130487 · Cited by 5,623
Performance of optical flow techniques
Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994) · International Journal of Computer Vision · doi:10.1007/bf01420984 · Cited by 4,305
LabelMe: A Database and Web-Based Tool for Image Annotation
Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0090-8 · Cited by 4,210
Self-citationA Bayesian Hierarchical Model for Learning Natural Scene Categories
Li, F.-F., & Perona, P. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.16 · Cited by 3,613
Parameterisation of a stochastic model for human face identification
Samaria, F. S., & Harter, A. C. (2002) · Proceedings of 1994 IEEE Workshop on Applications of Computer Vision · doi:10.1109/acv.1994.341300 · Cited by 2,593
High Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,570
A Database and Evaluation Methodology for Optical Flow
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0390-2 · Cited by 2,205 · cited 2×
“Inspired by the recent advances in image alignment and scene parsing, we propose SIFT flow to establish the correspondence between images across scenes.”
BackgroundMethodCootes, T. F., Edwards, G. J., & Taylor, C. J. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0054760 · Cited by 2,195
Pictorial Structures for Object Recognition
Felzenszwalb, P. F., & Huttenlocher, D. P. (2005) · International Journal of Computer Vision · doi:10.1023/b:visi.0000042934.15159.49 · Cited by 2,180
Pérez, P., Gangnet, M., & Blake, A. (2003) · ACM SIGGRAPH 2003 Papers · doi:10.1145/1201775.882269 · Cited by 2,020
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921
Self-citationPérez, P., Gangnet, M., & Blake, A. (2003) · ACM Transactions on Graphics · doi:10.1145/882262.882269 · Cited by 1,773
Image Alignment and Stitching: A Tutorial
Szeliski, R. (2007) · Foundations and Trends in Computer Graphics and Vision · doi:10.1561/0600000009 · Cited by 1,593
The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields
Black, M. J., & Anandan, P. (1996) · Computer Vision and Image Understanding · doi:10.1006/cviu.1996.0006 · Cited by 1,570
Evaluation of Interest Point Detectors
Schmid, C., Mohr, R., & Bauckhage, C. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008199403446 · Cited by 1,484
The pyramid match kernel: discriminative classification with sets of image features
Grauman, K., & Darrell, T. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.239 · Cited by 1,481
Support vector machines for histogram-based image classification
Chapelle, O., Haffner, P., & Vapnik, V. N. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.788646 · Cited by 1,468
Freeman, W. T., Pasztor, E. C., & Carmichael, O. T. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026501619075 · Cited by 1,459
Local grayvalue invariants for image retrieval
Schmid, C., & Mohr, R. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.589215 · Cited by 1,432
Blobworld: image segmentation using expectation-maximization and its application to image querying
Carson, C., Belongie, S., Greenspan, H., & Malik, J. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1023800 · Cited by 1,410
Efficient Belief Propagation for Early Vision
Felzenszwalb, P. F., & Huttenlocher, D. P. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-7899-4 · Cited by 1,345
Support vector machine active learning for image retrieval
Tong, S., & Chang, E. (2001) · Proceedings of the ninth ACM international conference on Multimedia · doi:10.1145/500141.500159 · Cited by 1,328
Stereo matching using belief propagation
Sun, J., Zheng, N.-N., & Shum, H.-Y. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1206509 · Cited by 1,271
Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods
Bruhn, A., Weickert, J., & Schnörr, C. (2005) · International Journal of Computer Vision · doi:10.1023/b:visi.0000045324.43199.43 · Cited by 1,160
Hierarchical model-based motion estimation
Bergen, J. R., Anandan, P., Hanna, K. J., & Hingorani, R. (1992) · Lecture Notes in Computer Science · doi:10.1007/3-540-55426-2_27 · Cited by 1,154
Computing visual correspondence with occlusions using graph cuts
Kolmogorov, V., & Zabih, R. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937668 · Cited by 1,119
Avidan, S. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.35 · Cited by 1,112
Alignment by maximization of mutual information
Viola, P., & Wells, W. M. (2002) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466930 · Cited by 1,082
Matching Local Self-Similarities across Images and Videos
Shechtman, E., & Irani, M. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383198 · Cited by 1,076
Kernel Methods in Computational Biology
(2004) · The MIT Press eBooks · doi:10.7551/mitpress/4057.001.0001 · Cited by 1,032
Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., & Rother, C. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70844 · Cited by 966
Lucas/Kanade meets Horn/Schunck : combining local and global optic flow methods
Weickert, J., Bruhn, A., & Schnörr, C. (2003) · doi:10.22028/d291-26234 · Cited by 963
Shape Matching and Object Recognition Using Low Distortion Correspondences
Berg, A. C., Berg, T. L., & Malik, J. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.320 · Cited by 845
Scene completion using millions of photographs
Hays, J., & Efros, A. A. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276382 · Cited by 687
A Database and Evaluation Methodology for Optical Flow
Baker, S., Roth, S., Scharstein, D., Black, M. J., Lewis, J. P., & Szeliski, R. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408903 · Cited by 618
SIFT Flow: Dense Correspondence across Different Scenes
Liu, C., Yuen, J., Torralba, A., Sivic, J., & Freeman, W. T. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88690-7_3 · Cited by 540
Self-citationCosegmentation of Image Pairs by Histogram Matching - Incorporating a Global Constraint into MRFs
Rother, C., Minka, T., Blake, A., & Kolmogorov, V. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.91 · Cited by 535
Scene completion using millions of photographs
Hays, J., & Efros, A. A. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276382 · Cited by 534
Computational Experiments with a Feature Based Stereo Algorithm
Grimson, W. E. L. (1985) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1985.4767615 · Cited by 529
LOCUS: learning object classes with unsupervised segmentation
Winn, J., & Jojic, N. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.148 · Cited by 520
Shape Context: A New Descriptor for Shape Matching and Object Recognition
Belongie, S., Malik, J., & Puzicha, J. (2000) · Cited by 511
A survey of kernels for structured data
Gärtner, T. (2003) · ACM SIGKDD Explorations Newsletter · doi:10.1145/959242.959248 · Cited by 452
Learning a Spatially Smooth Subspace for Face Recognition
Cai, D., He, X., Hu, Y., Han, J., & Huang, T. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383054 · Cited by 376
Recognition with local features: the kernel recipe
Wallraven, Caputo, & Graf (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238351 · Cited by 347
Online handwriting recognition with support vector machines - a kernel approach
Bahlmann, C., Haasdonk, B., & Bürkhardt, H. (2003) · International Workshop on Frontiers in Handwriting Recognition (IWFHR) · doi:10.1109/iwfhr.2002.1030883 · Cited by 330
Nonparametric scene parsing: Label transfer via dense scene alignment
Liu, C., Yuen, J., & Torralba, A. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206536 · Cited by 315
Self-citationData Fusion and Multicue Data Matching by Diffusion Maps
Lafon, S., Keller, Y., & Coifman, R. R. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.223 · Cited by 312
Nie, J.-Y., Simard, M., Isabelle, P., & Durand, R. (1999) · Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/312624.312656 · Cited by 306
Phase-based disparity measurement
Fleet, D. J., Jepson, A. D., & Jenkin, M. R. M. (1991) · CVGIP: Image Understanding · doi:10.1016/1049-9660(91)90027-m · Cited by 293
Large displacement optical flow
Brox, T., Bregler, C., & Malik, J. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206697 · Cited by 277
Registration of Challenging Image Pairs: Initialization, Estimation, and Decision
Yang, G., Stewart, C. V., Sofka, M., & Tsai, C.-L. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1116 · Cited by 259
Smoothness in layers: Motion segmentation using nonparametric mixture estimation
Weiss, Y. (2002) · Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1997.609375 · Cited by 208
Computational framework for determining stereo correspondence from a set of linear spatial filters
Jones, D. G., & Malik, J. (1992) · Image and Vision Computing · doi:10.1016/0262-8856(92)90015-u · Cited by 191
Texture orientation for sorting photos "at a glance"
Gorkani, M. M., & Picard, R. W. (2002) · Proceedings of 12th International Conference on Pattern Recognition · doi:10.1109/icpr.1994.576325 · Cited by 187
Human-assisted motion annotation
Liu, C., Freeman, W. T., Adelson, E. H., & Weiss, Y. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587845 · Cited by 186
Self-citationComputer Matching of Areas in Stereo Images
Hannah, M. J. (1974) · doi:10.21236/ad0786720 · Cited by 184
Mercer Kernels for Object Recognition with Local Features
Lyu, S. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.223 · Cited by 152
Consensus Network Decoding for Statistical Machine Translation System Combination
Sim, K. C., Byrne, B., Gales, M., Sahbi, H., & Woodland, P. C. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.367174 · Cited by 105
Object Recognition by Scene Alignment
Russell, B. C., Torralba, A. B., Liu, C., Fergus, R., & Freeman, W. T. (2007) · Cited by 104
Self-citationNg, J., & Gong, S. (2003) · Analysis and Modeling of Faces and Gestures · doi:10.1109/ratfg.1999.799218 · Cited by 64
Efficient MRF Deformation Model for Non-Rigid Image Matching
Shekhovtsov, A., Kovtun, I., & Hlaváč, V. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383205 · Cited by 59
Real-Time Connectivity Constrained Depth Map Computation Using Programmable Graphics Hardware
Cornelis, N., & Gool, L. V. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.291 · Cited by 46
Context-dependent kernel design for object matching and recognition
Sahbi, H., Audibert, J.-Y., Rabarisoa, J., et al. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587607 · Cited by 35
Hausdorff Kernel for 3D Object Acquisition and Detection
Barla, A., Odone, F., & Verri, A. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47979-1_2 · Cited by 28
Cited by
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Lin, G., Milan, A., Shen, C., & Reid, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.549 · Cited by 3,287
“[32] RGB-D − 10.”
SUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943
LoFTR: Detector-Free Local Feature Matching with Transformers
Sun, J., Shen, Z., Wang, Y., Bao, H., & Zhou, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00881 · Cited by 1,719
Metadata
- Authors
- Liu, C., Yuen, J., & Torralba, A.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2011-05
- Volume
- 33
- Issue
- 5
- Pages
- 978-994
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2090518410
- Cited by
- 1,582
- References
- 78
- FWCI
- 54.18
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.