The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval
Abstract
The problem of cross-modal retrieval from multimedia repositories is considered. This problem addresses the design of retrieval systems that support queries across content modalities, for example, using an image to search for texts. A mathematical formulation is proposed, equating the design of cross-modal retrieval systems to that of isomorphic feature spaces for different content modalities. Two hypotheses are then investigated regarding the fundamental attributes of these spaces. The first is that low-level cross-modal correlations should be accounted for. The second is that the space should enable semantic abstraction. Three new solutions to the cross-modal retrieval problem are then derived from these hypotheses: correlation matching (CM), an unsupervised method which models cross-modal correlations, semantic matching (SM), a supervised technique that relies on semantic representation, and semantic correlation matching (SCM), which combines both. An extensive evaluation of retrieval performance is conducted to test the validity of the hypotheses. All approaches are shown successful for text retrieval in response to image queries and vice versa. It is concluded that both hypotheses hold, in a complementary form, although evidence in favor of the abstraction hypothesis is stronger than that for correlation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Jolliffe, I. (2005) · Encyclopedia of Statistics in Behavioral Science · doi:10.1002/0470013192.bsa501 · Cited by 14,310
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Introduction to Modern Information Retrieval
Salton, G., & McGill, M. J. (1983) · Virtual Defense Library (Ministerio de Defensa) · Cited by 11,390
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Kernel Methods for Pattern Analysis
Shawe-Taylor, J., & Cristianini, N. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9780511809682 · Cited by 6,587
Content-based image retrieval at the end of the early years
Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895972 · Cited by 6,056
Introduction to modern information retrieval
Dillon, M. (1983) · Information Processing & Management · doi:10.1016/0306-4573(83)90062-6 · Cited by 5,909
Swain, M. J., & Ballard, D. H. (1991) · International Journal of Computer Vision · doi:10.1007/bf00130487 · Cited by 5,623
Cited by
Adversarial Cross-Modal Retrieval
Wang, B., Yang, Y., Xu, X., Hanjalic, A., & Shen, H. T. (2017) · Proceedings of the 25th ACM international conference on Multimedia · doi:10.1145/3123266.3123326 · Cited by 779
Semantics-preserving hashing for cross-view retrieval
Lin, Z., Ding, G., Hu, M., & Wang, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299011 · Cited by 542
A Survey of Multi-View Representation Learning
Li, Y., Yang, M., & Zhang, Z. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2872063 · Cited by 524
Deep Supervised Cross-Modal Retrieval
Zhen, L., Hu, P., Wang, X., & Peng, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01064 · Cited by 478
Deep Collaborative Embedding for Social Image Understanding
Li, Z., Tang, J., & Mei, T. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2852750 · Cited by 404
Cross-Modal Retrieval With CNN Visual Features: A New Baseline
Wei, Y., Zhao, Y., Lu, C., Wei, S., Liu, L., Zhu, Z., & Yan, S. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2519449 · Cited by 370
Deep Joint-Semantics Reconstructing Hashing for Large-Scale Unsupervised Cross-Modal Retrieval
Su, S., Zhong, Z., & Zhang, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00312 · Cited by 342
An Overview of Cross-Media Retrieval: Concepts, Methodologies, Benchmarks, and Challenges
Peng, Y., Huang, X., & Zhao, Y. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2705068 · Cited by 341
Joint Feature Selection and Subspace Learning for Cross-Modal Retrieval
Wang, K., He, R., Wang, L., Wang, W., & Tan, T. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2505311 · Cited by 318
Image Captioning with Deep Bidirectional LSTMs
Wang, C., Yang, H., Bartz, C., & Meinel, C. (2016) · Proceedings of the 24th ACM international conference on Multimedia · doi:10.1145/2964284.2964299 · Cited by 287
Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding
Zhang, L., Zhang, Q., Zhang, L., Tao, D., Huang, X., & Du, B. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2014.12.016 · Cited by 278
HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
Vulić, I., Gerz, D., Kiela, D., Hill, F., & Korhonen, A. (2016) · arXiv · doi:10.48550/arxiv.1608.02117 · Cited by 90
Metadata
- Authors
- Costa Pereira, J., Coviello, E., Doyle, G., Rasiwasia, N., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2014-03
- Volume
- 36
- Issue
- 3
- Pages
- 521-535
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2138118304
- Cited by
- 435
- References
- 62
- FWCI
- 35.31
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.