The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Summarizing visual data using bidirectional similarity
Abstract
We propose a principled approach to summarization of visual data (images or video) based on optimization of a well-defined similarity measure. The problem we consider is re-targeting (or summarization) of image/video data into smaller sizes. A good ldquovisual summaryrdquo should satisfy two properties: (1) it should contain as much as possible visual information from the input data; (2) it should introduce as few as possible new visual artifacts that were not in the input data (i.e., preserve visual coherence). We propose a bi-directional similarity measure which quantitatively captures these two requirements: Two signals S and T are considered visually similar if all patches of S (at multiple scales) are contained in T, and vice versa. The problem of summarization/re-targeting is posed as an optimization problem of this bi-directional similarity measure. We show summarization results for image and video data. We further show that the same approach can be used to address a variety of other problems, including automatic cropping, completion and synthesis of visual data, image collage, object removal, photo reshuffling and more.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Texture synthesis by non-parametric sampling
Efros, A. A., & Leung, T. K. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790383 · Cited by 3,023
Seam carving for content-aware image resizing
Avidan, S., & Shamir, A. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276390 · Cited by 1,362
Space-Time Completion of Video
Wexler, Y., Shechtman, E., & Irani, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.60 · Cited by 682
Self-citationNon-homogeneous Content-driven Video-retargeting
Wolf, L., Guttmann, M., & Cohen-Or, D. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409010 · Cited by 503
Automatic thumbnail cropping and its effectiveness
Suh, B., Ling, H., Bederson, B. B., & Jacobs, D. W. (2003) · Proceedings of the 16th annual ACM symposium on User interface software and technology · doi:10.1145/964696.964707 · Cited by 421
A visual attention model for adapting images on small displays
Chen, L.-Q., Xie, X., Fan, X., Ma, W.-Y., Zhang, H.-J., & Zhou, H.-Q. (2003) · Multimedia Systems · doi:10.1007/s00530-003-0105-4 · Cited by 419
Setlur, V., Takagi, S., Raskar, R., Gleicher, M., & Gooch, B. (2005) · Proceedings of the 4th international conference on Mobile and ubiquitous multimedia - MUM '05 · doi:10.1145/1149488.1149499 · Cited by 220
Epitomic analysis of appearance and shape
Jojic, Frey, & Kannan (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238311 · Cited by 218
Video retargeting: automating pan and scan
Liu, F., & Gleicher, M. (2006) · Proceedings of the 14th ACM international conference on Multimedia · doi:10.1145/1180639.1180702 · Cited by 194
Automatic image retargeting with fisheye-view warping
Liu, F., & Gleicher, M. (2005) · Proceedings of the 18th annual ACM symposium on User interface software and technology · doi:10.1145/1095034.1095061 · Cited by 179
Cited by
PatchMatch: a randomized correspondence algorithm for structural image editing
Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531330 · Cited by 2,880
Self-citationGenerative Image Inpainting with Contextual Attention
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00577 · Cited by 2,527
Image Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419
“However, these steps often come at a large computation cost such as in [26].”
BackgroundGlobally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203
Free-Form Image Inpainting With Gated Convolution
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00457 · Cited by 1,928
Super-resolution from a single image
Glasner, D., Bagon, S., & Irani, M. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459271 · Cited by 1,889
“In the classical multi-image SR (e.g., [12, 5, 8] to name just a few) a set of low-resolution images of the same scene are taken (at subpixel misalignments).”
BackgroundSelf-citationVisual saliency based on multiscale deep features
Li, G., & Yu, Y. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299184 · Cited by 1,163
SinGAN: Learning a Generative Model From a Single Natural Image
Shaham, T. R., Dekel, T., & Michaeli, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00467 · Cited by 890
DHSNet: Deep Hierarchical Saliency Network for Salient Object Detection
Liu, N., & Han, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.80 · Cited by 833
Salient Object Detection in the Deep Learning Era: An In-Depth Survey
Wang, W., Lai, Q., Fu, H., Shen, J., Ling, H., & Yang, R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3051099 · Cited by 811
Discovering important people and objects for egocentric video summarization
Lee, Y. J., Ghosh, J., & Grauman, K. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247820 · Cited by 658
Image melding: combining inconsistent images using patch-based synthesis
Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., & Sen, P. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185578 · Cited by 550
Self-citationThe Generalized PatchMatch Correspondence Algorithm
Barnes, C., Shechtman, E., Goldman, D. B., & Finkelstein, A. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15558-1_3 · Cited by 530
Self-citationShift-Net: Image Inpainting via Deep Feature Rearrangement
Yan, Z., Li, X., Li, M., Zuo, W., & Shan, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_1 · Cited by 511
Deep Saliency with Encoded Low Level Distance Map and High Level Features
Lee, G., Tai, Y.-W., & Kim, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.78 · Cited by 487
Visual attribute transfer through deep image analogy
Liao, J., Yao, Y., Yuan, L., Hua, G., & Kang, S. B. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073683 · Cited by 487
Robust patch-based hdr reconstruction of dynamic scenes
Sen, P., Kalantari, N. K., Yaesoubi, M., Darabi, S., Goldman, D. B., & Shechtman, E. (2012) · ACM Transactions on Graphics · doi:10.1145/2366145.2366222 · Cited by 433
Self-citationState of the Art on Neural Rendering
Tewari, A., Fried, O., Thies, J., Sitzmann, V., Lombardi, S., Sunkavalli, K., Martin‐Brualla, R., Simon, T., Saragih, J., Nießner, M., et al. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14022 · Cited by 430
Self-citationMulti-operator media retargeting
Rubinstein, M., Shamir, A., & Avidan, S. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531329 · Cited by 419
Foreground-Aware Image Inpainting
Xiong, W., Yu, J., Lin, Z., Yang, J., Lu, X., Barnes, C., & Luo, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00599 · Cited by 402
Prit, Y., Kav-Venaki, E., & Peleg, S. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459159 · Cited by 400
StructureFlow: Image Inpainting via Structure-Aware Appearance Flow
Ren, Y., Yu, X., Zhang, R., Li, T. H., Liu, S., & Li, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00027 · Cited by 396
Visual Saliency Detection Based on Multiscale Deep CNN Features
Li, G., & Yu, Y. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2602079 · Cited by 394
Coherent Semantic Attention for Image Inpainting
Liu, H., Jiang, B., Xiao, Y., & Yang, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00427 · Cited by 383
PatchMatch: a randomized correspondence algorithm for structural image editing
Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009) · ACM SIGGRAPH 2009 papers · doi:10.1145/1576246.1531330 · Cited by 364
Self-citationRobust Multi-Exposure Image Fusion: A Structural Patch Decomposition Approach
Ma, K., Li, H., Yong, H., Wang, Z., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2671921 · Cited by 355
“Double-DIP”: Unsupervised Image Decomposition via Coupled Deep-Image-Priors
Gandelsman, Y., Shocher, A., & Irani, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01128 · Cited by 355
Self-citationA comparative study of image retargeting
Rubinstein, M., Gutiérrez, D., Sorkine‐Hornung, O., & Shamir, A. (2010) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/1866158.1866186 · Cited by 343
A framework for visual saliency detection with applications to image thumbnailing
Marchesotti, L., Cifarelli, C., & Csurka, G. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459467 · Cited by 322
PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing
Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596777 · Cited by 276
Self-citationLiu, L., Chen, R., Wolf, L., & Cohen‐Or, D. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01616.x · Cited by 255
Metadata
- Authors
- Simakov, D., Caspi, Y., Shechtman, E., & Irani, M.
- Venue
- 2008 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2008-06
- Pages
- 1-8
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2115273023
- Cited by
- 613
- References
- 15
- FWCI
- 29.12
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.