Content-based multimedia information retrieval: State of the art and challenges
The DOI matches its Crossref record; 150 of the 152 references it cites were found in a registry or an index.
Abstract
Extending beyond the boundaries of science, art, and culture, content-based multimedia information retrieval provides new paradigms and methods for searching through the myriad variety of media all over the world. This survey reviews 100+ recent articles on content-based multimedia information retrieval and discusses their role in current research directions which include browsing and search paradigms, user studies, affective computing, learning, semantic queries, new features and media types, high performance indexing, and evaluation techniques. Based on the current state of the art, we discuss the major challenges for the future.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similarly, given a diagnostically challenging WSI, retrieving a similar WSI via slide search5,46–53 or pathology reports through cross-modal report search10,54–56 typically requires specialized algorithms to bridge the gap between fine-grained patch embeddings and slide-level information, introducing hurdles to clinical adoption.”
From A multimodal whole-slide foundation model for pathology (2025) · cites this paper as [46]
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
A Combined Corner and Edge Detector
Harris, C., & Stephens, M. (1988) · Procedings of the Alvey Vision Conference 1988 · doi:10.5244/c.2.23 · Cited by 12,901
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
A comparative study of texture measures with classification based on featured distributions
Ojala, T., Pietikäinen, M., & Harwood, D. (1996) · Pattern Recognition · doi:10.1016/0031-3203(95)00067-4 · Cited by 6,882
Content-based image retrieval at the end of the early years
Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895972 · Cited by 6,056
Self-citation2009 IEEE Conference on Computer Vision and Pattern Recognition
(2009) · doi:10.1109/cvpr14067.2009 · Cited by 5,550
Ballard, D. H., & Brown, C. M. (1982) · Cited by 5,303
Haralock, R. M., & Shapiro, L. G. (1991) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 3,933
Scale & Affine Invariant Interest Point Detectors
Mikolajczyk, K., & Schmid, C. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000027790.02288.f2 · Cited by 3,436
Detecting faces in images: a survey
Yang, M.-H., Kriegman, D. J., & Ahuja, N. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.982883 · Cited by 3,427
Research works printed but not linked: 5
- Bach , J. R. , Fuller , C. , Gupta , A. , Hampapur , A. , Horowitz , B. , Humphrey , R. , Jain , R. , and Shu , C. F . 1996. Virage image search engine: An open framework for image management . In Proceedings of the SPIE Storage and Retrieval for Still Image and Video Databases. 76--87 .]] Bach, J. R., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Humphrey, R., Jain, R., and Shu, C. F. 1996. Virage image search engine: An open framework for image management. In Proceedings of the SPIE Storage and Retrieval for Still Image and Video Databases. 76--87.]]
- Eiter T. and Libkin L. 2005. Database Theory. Springer London. UK.]] Eiter T. and Libkin L. 2005. Database Theory. Springer London. UK.]]
- Enser , P. G. B. and Sandom , C. J . 2003. Towards a comprehensive survey of the semantic gap in visual information retrieval . In Proceedings of the 2nd International Conference on Image and Video Retrieval. (July) , Urbana, IL, E. M. Bakker, T. S. Huang, M. S. Lew, N. Sebe and X. Zhou, Eds. Springer-Verlag, London, UK. 291--299.]] Enser, P. G. B. and Sandom, C. J. 2003. Towards a comprehensive survey of the semantic gap in visual information retrieval. In Proceedings of the 2nd International Conference on Image and Video Retrieval. (July), Urbana, IL, E. M. Bakker, T. S. Huang, M. S. Lew, N. Sebe and X. Zhou, Eds. Springer-Verlag, London, UK. 291--299.]]
- Lew , M. S. 2001. Principles of Visual Information Retrieval . Springer , London, UK .]] Lew, M. S. 2001. Principles of Visual Information Retrieval. Springer, London, UK.]]
- Tian , Q. , Moghaddam , B. , and Huang , T. S . 2002. Visualization, estimation and user-modeling . In Proceedings of the 1st International Conference on Image and Video Retrieval. (July) , London, UK, M. S. Lew, N. Sebe, and J. P. Eakins, Eds. Springer-Verlag, London, UK, 7--16.]] Tian, Q., Moghaddam, B., and Huang, T. S. 2002. Visualization, estimation and user-modeling. In Proceedings of the 1st International Conference on Image and Video Retrieval. (July), London, UK, M. S. Lew, N. Sebe, and J. P. Eakins, Eds. Springer-Verlag, London, UK, 7--16.]]
Cited by
Image retrieval: Ideas, influences, and trends of the new age
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008) · ACM Computing Surveys · doi:10.1145/1348246.1348248 · Cited by 3,040
Chen, M., Mao, S., & Liu, Y. (2014) · Mobile Networks and Applications · doi:10.1007/s11036-013-0489-0 · Cited by 2,913
Image-Based Recommendations on Styles and Substitutes
McAuley, J., Targett, C., Shi, Q., & van den Hengel, A. (2015) · Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/2766462.2767755 · Cited by 2,280
The MIR flickr retrieval evaluation
Huiskes, M. J., & Lew, M. S. (2008) · Proceedings of the 1st ACM international conference on Multimedia information retrieval · doi:10.1145/1460096.1460104 · Cited by 1,836
Self-citationHow to Evaluate Foreground Maps
Margolin, R., Zelnik-Manor, L., & Tal, A. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.39 · Cited by 1,106
Metadata
- Authors
- Lew, M. S., Sebe, N., Djeraba, C., & Jain, R.
- Affiliations
- Leiden University (NL)
- University of Amsterdam (NL)
- Laboratoire d'Informatique Fondamentale de Lille (FR)
- University of California, Irvine (US)
- Venue
- ACM Transactions on Multimedia Computing, Communications, and Applications
- Published
- 2006-02
- Volume
- 2
- Issue
- 1
- Pages
- 1-19
- ISSN
- 1551-6857 (ISSN-L)
- 1551-6865
- OpenAlex
- W2147069236
- Semantic Scholar
- cabe73e89b6389a163393ecbf1b82d20ba4ea10f
- Cited by
- 1,528
- References
- 152
- Citations by year
- 2026: 17 · 2025: 13 · 2024: 20 · 2023: 25 · 2022: 18 · 2021: 31 · 2020: 34 · 2019: 58 · 2018: 64 · 2017: 88 · 2016: 77 · 2015: 102 · 2014: 113 · 2013: 132 · 2012: 160
- FWCI
- 113.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Multimedia Computing Communications and Applications (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- HAL (submittedVersion)
- Iris (University of Trento) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- multimedia browsing, multimedia retrieval, user study, affective computing, multimedia learning, semantic queries, indexing, evaluation techniques, media types
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Content-based multimedia information retrieval: State of the art and challenges. https://app.cochat.ai/papers/10.1145/1126004.1126005
- The paper
- Lew, M. S., Sebe, N., Djeraba, C., & Jain, R. (2006). Content-based multimedia information retrieval: State of the art and challenges. ACM Transactions on Multimedia Computing, Communications, and Applications. https://doi.org/10.1145/1126004.1126005
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.