The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Storytelling with Image Data: A Systematic Review and Comparative Analysis of Methods and Tools
Abstract
In our digital age, data are generated constantly from public and private sources, social media platforms, and the Internet of Things. A significant portion of this information comes in the form of unstructured images and videos, such as the 95 million daily photos and videos shared on Instagram and the 136 billion images available on Google Images. Despite advances in image processing and analytics, the current state of the art lacks effective methods for discovering, linking, and comprehending image data. Consider, for instance, the images from a crime scene that hold critical information for a police investigation. Currently, no system can interactively generate a comprehensive narrative of events from the incident to the conclusion of the investigation. To address this gap in research, we have conducted a thorough systematic literature review of existing methods, from labeling and captioning to extraction, enrichment, and transforming image data into contextualized information and knowledge. Our review has led us to propose the vision of storytelling with image data, an innovative framework designed to address fundamental challenges in image data comprehension. In particular, we focus on the research problem of understanding image data in general and, specifically, curating, summarizing, linking, and presenting large amounts of image data in a digestible manner to users. In this context, storytelling serves as an appropriate metaphor, as it can capture and depict the narratives and insights locked within the relationships among data stored across different islands. Additionally, a story can be subjective and told from various perspectives, ranging from a highly abstract narrative to a highly detailed one.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“At the same time, online platforms and cloud-based storage systems have transformed photo preservation and sharing practices (Li et al., 2019a;Lotfi et al., 2023), moving from physical albums to digital ecosystems (Zhang et al., 2025).”
From Designing for Reminiscence in Digital Photo Systems for Older Adults: A Scoping Review (2026) · cites this paper as Lotfi et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
SURF: Speeded Up Robust Features
Bay, H., Tuytelaars, T., & Van Gool, L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_32 · Cited by 14,955 · cited 1×
“To be able to detect the objects without the rotation and scaling concerns, intersect point and local features detectors such as Scale-Invariant Feature Transform (SIFT) [42] and Speeded Up Robust Features (SURF) [43] could be used.”
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557 · cited 1×
“The traditional approaches use well-established computer vision (CV) techniques such as feature descriptors (e.g., SIFT [2], SURF [3], BRIEF [4]), and after that, a feature extraction step is carried out.”
EfficientDet: Scalable and Efficient Object Detection
Tan, M., Pang, R., & Le, Q. V. (2019) · arXiv · Cited by 9,841
Research works printed but not linked: 169
- Beheshti, A., Ghodratnama, S., Elahi, M., and Farhood, H. (2022). Social Data Analytics, CRC Press. · doi:10.1201/9781003260141
- Lindeberg, T. (2012). Scale Invariant Feature Transform, KTH. · DOI · Paper page
- Calonder, M., Lepetit, V., Strecha, C., and Fua, P. (2010, January 5–11). Brief: Binary robust independent elementary features. Proceedings of the European Conference on Computer Vision, Heraklion, Greece. · DOI · Paper page
- Li, Q., Li, J., Sheng, J., Cui, S., Wu, J., Hei, Y., Peng, H., Guo, S., Wang, L., and Beheshti, A. (IEEE Trans. Neural Netw. Learn. Syst., 2022). A Survey on Deep Learning Event Extraction: Approaches and Applications, IEEE Trans. Neural Netw. Learn. Syst., Early Access. · DOI
- Beheshti, A., Tabebordbar, A., and Benatallah, B. (2020, January 20–24). istory: Intelligent storytelling with social data. Proceedings of the Companion Proceedings of the Web Conference 2020, Taipei, Taiwan. · doi:10.1145/3366424.3383553
- Thöny, M., Schnürer, R., Sieber, R., Hurni, L., and Pajarola, R. (2018). Storytelling in interactive 3D geographic visualization systems. ISPRS Int. J. Geo-Inf., 7. · doi:10.3390/ijgi7030123
- Beheshti, A. (2022, January 11–15). Knowledge base 4.0: Using crowdsourcing services for mimicking the knowledge of domain experts. Proceedings of the 2022 IEEE International Conference on Web Services (ICWS), Barcelona, Spain. · DOI
- Beheshti, A., Benatallah, B., Sheng, Q.Z., and Schiliro, F. (2020, January 20–24). Intelligent knowledge lakes: The age of artificial intelligence and big data. Proceedings of the International Conference on Web Information Systems Engineering, Amsterdam, The Netherlands. · doi:10.1007/978-981-15-3281-8_3
- Zhang, 2019, Knowledge Graph-based image classification refinement, IEEE Access, 7, 57678, 10.1109/ACCESS.2019.2912627 · DOI
- Gong, W., Zhang, X., Chen, Y., He, Q., Beheshti, A., Xu, X., Yan, C., and Qi, L. (2022, January 11–15). DAWAR: Diversity-aware web APIs recommendation for mashup creation based on correlation graph. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain. · DOI
- Sagheer, 2020, A review on medical image denoising algorithms, Biomed. Signal Process. Control, 61, 102036, 10.1016/j.bspc.2020.102036 · DOI · Paper page
- Brooks, T., Mildenhall, B., Xue, T., Chen, J., Sharlet, D., and Barron, J.T. (2019, January 16–17). Unprocessing images for learned raw denoising. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA. · DOI · Paper page
- Saafin, W., and Schaefer, G. (2017, January 11–13). Pre-processing techniques for colour digital pathology image analysis. Proceedings of the Annual Conference on Medical Image Understanding and Analysis, Edinburgh, UK. · DOI
- Krig, S. (2016). Computer Vision Metrics, Springer. · DOI
- Heilbronner, R., and Barrett, S. (2014). Image Analysis in Earth Sciences, Springer. · doi:10.1007/978-3-642-10343-8
- Wang, Y., Cao, Y., Zha, Z.J., Zhang, J., Xiong, Z., Zhang, W., and Wu, F. (2019, January 21–25). Progressive retinex: Mutually reinforced illumination-noise perception network for low-light image enhancement. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France. · DOI
- Heilbronner, R., and Barrett, S. (2013). Image Analysis in Earth Sciences: Microstructures and Textures of Earth Materials, Springer Science & Business Media. · doi:10.1007/978-3-642-10343-8
- Kim, 2021, Low-light image enhancement by diffusion pyramid with residuals, J. Vis. Commun. Image Represent., 81, 103364, 10.1016/j.jvcir.2021.103364 · DOI
- Pang, B., Zhai, D., Jiang, J., and Liu, X. (2020, January 12–16). Single image deraining via scale-space invariant attention neural network. Proceedings of the 28th ACM International Conference on Multimedia, Virtual/Seattle, WA, USA. · DOI
- Liu, 2020, Deep learning–based enhancement of motion blurred UAV concrete crack images, J. Comput. Civ. Eng., 34, 04020028, 10.1061/(ASCE)CP.1943-5487.0000907 · DOI
- Bai, 2019, Imaging enhancement of light-sheet fluorescence microscopy via deep learning, IEEE Photonics Technol. Lett., 31, 1803, 10.1109/LPT.2019.2948030 · DOI
- Dong, 2007, Image integration in electroanatomic mapping, Herzschrittmachertherapie Elektrophysiologie, 18, 122, 10.1007/s00399-007-0571-z · doi:10.1007/s00399-007-0571-z
- Zach, C., Pock, T., and Bischof, H. (2007, January 14–21). A globally optimal algorithm for robust tv-l 1 range image integration. Proceedings of the 2007 IEEE 11th International Conference on Computer Vision, Rio De Janeiro, Brazil. · DOI · Paper page
- Dogra, 2020, An efficient image integration algorithm for night mode vision applications, Multimed. Tools Appl., 79, 10995, 10.1007/s11042-018-6631-z · doi:10.1007/s11042-018-6631-z
- Bavirisetti, 2018, Multi-focus image fusion using multi-scale image decomposition and saliency detection, Ain Shams Eng. J., 9, 1103, 10.1016/j.asej.2016.06.011 · doi:10.1016/j.asej.2016.06.011
- Wang, 2018, Perceptual adversarial networks for image-to-image transformation, IEEE Trans. Image Process., 27, 4066, 10.1109/TIP.2018.2836316 · DOI · Paper page
- Sarid, 2011, Image formation and image transformation, Arts Psychother., 38, 252, 10.1016/j.aip.2011.07.001 · DOI
- Jia, 2012, Image transformation based on learning dictionaries across image spaces, IEEE Trans. Pattern Anal. Mach. Intell., 35, 367, 10.1109/TPAMI.2012.95 · DOI
- Vial, 2018, The role of deep learning and radiomic feature extraction in cancer-specific predictive modelling: A review, Transl. Cancer Res., 7, 803, 10.21037/tcr.2018.05.02 · DOI · Paper page
- Tofighi, G., Venetsanopoulos, A.N., Raahemifar, K., Beheshti, S., and Mohammadi, H. (2013, January 1–3). Hand posture recognition using K-NN and Support Vector Machine classifiers evaluated on our proposed HandReader dataset. Proceedings of the 2013 18th International Conference on Digital Signal Processing (DSP), Fira, Greece. · DOI
- Zhao, 2021, Fingerprint pre-processing and feature engineering to enhance agricultural products categorization, Future Gener. Comput. Syst., 125, 944, 10.1016/j.future.2021.07.005 · doi:10.1016/j.future.2021.07.005
- Wiggers, K.L., Britto, A.S., Heutte, L., Koerich, A.L., and Oliveira, L.E.S. (2018, January 8–13). Document image retrieval using deep features. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil. · DOI
- Farhood, H., He, X., Jia, W., Blumenstein, M., and Li, H. (December, January 29). Counting people based on linear, weighted, and local random forests. Proceedings of the 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Sydney, Australia. · DOI
- Singh, A., and Sharma, D.K. (2020). Data Visualization and Knowledge Engineering, Springer.
- Zou, X. (2019, January 10–11). A Review of object detection techniques. Proceedings of the 2019 International Conference on Smart Grid and Electrical Automation (ICSGEA), Xiangtan, China. · DOI
- Ballard, 1981, Generalizing the Hough transform to detect arbitrary shapes, Pattern Recognit., 13, 111, 10.1016/0031-3203(81)90009-1 · DOI · Paper page
- Harris, 1988, A combined corner and edge detector, Proceedings of the Alvey Vision Conference, Volume 15, 1
- He, 2015, Spatial pyramid pooling in deep convolutional networks for visual recognition, IEEE Trans. Pattern Anal. Mach. Intell., 37, 1904, 10.1109/TPAMI.2015.2389824 · DOI · Paper page
- Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile. · DOI · Paper page
- Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA. · DOI · Paper page
- Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands. · DOI · Paper page
- Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy. · DOI · Paper page
- Dash, 2020, Topic-based image caption generation, Arab. J. Sci. Eng., 45, 3025, 10.1007/s13369-019-04262-2 · DOI
- Guo, H., Zheng, K., Fan, X., Yu, H., and Wang, S. (2019, January 15–20). Visual attention consistency under image transforms for multi-label image classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA. · DOI · Paper page
- Ghiasi, G., Lin, T.Y., and Le, Q.V. (2019, January 15–20). Nas-fpn: Learning scalable feature pyramid architecture for object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA. · DOI · Paper page
- Lotfi, F., Jamzad, M., and Beigy, H. (2021, January 28–29). Automatic Image Annotation using Tag Relations and Graph Convolutional Networks. Proceedings of the 2021 5th International Conference on Pattern Recognition and Image Analysis (IPRIA), Kashan, Iran. · DOI
- Joseph, K., Khan, S., Khan, F.S., and Balasubramanian, V.N. (2021, January 19–25). Towards open world object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA. · DOI · Paper page
- Lotfi, F., Jamzad, M., and Beigy, H. (December, January 29). Automatic Image Annotation Using Quantization Reweighting Function and Graph Neural Networks. Proceedings of the International Conference on Service-Oriented Computing, Seville, Spain.
- Yang, P., Luo, F., Chen, P., Li, L., Yin, Z., He, X., and Sun, X. (2019, January 11–12). Knowledgeable Storyteller: A Commonsense-Driven Generative Model for Visual Storytelling. Proceedings of the IJCAI, Macao, China. · DOI
- Speer, R., and Havasi, C. (2012, January 23–25). Representing general relational knowledge in conceptnet 5. Proceedings of the LREC, Istanbul, Turkey.
- Chen, H., Huang, Y., Takamura, H., and Nakayama, H. (2021, January 2–9). Commonsense knowledge aware concept selection for diverse and informative visual storytelling. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual. · DOI
- Li, J., Shi, H., Tang, S., Wu, F., and Zhuang, Y. (2019, January 21–25). Informative visual storytelling with cross-modal rules. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France. · DOI
- Aggarwal, C.C. (2015). Data Mining, Springer. · DOI · Paper page
- Ahmed, 2019, Data summarization: A survey, Knowl. Inf. Syst., 58, 249, 10.1007/s10115-018-1183-0 · DOI
- Mao, J., Xu, W., Yang, Y., Wang, J., Huang, Z., and Yuille, A. (2014). Deep captioning with multimodal recurrent neural networks (m-rnn). arXiv.
- Karpathy, A., and Fei-Fei, L. (2015, January 7–12). Deep visual-semantic alignments for generating image descriptions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
- Vinyals, O., Toshev, A., Bengio, S., and Erhan, D. (2015, January 7–12). Show and tell: A neural image caption generator. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
- Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., and Goel, V. (2017, January 21–26). Self-critical sequence training for image captioning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA. · DOI · Paper page
- Ren, Z., Wang, X., Zhang, N., Lv, X., and Li, L.J. (2017, January 21–26). Deep reinforcement learning-based image captioning with embedding reward. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA. · DOI · Paper page
- Gordon, D., Kembhavi, A., Rastegari, M., Redmon, J., Fox, D., and Farhadi, A. (2018, January 18–22). Iqa: Visual question answering in interactive environments. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. · DOI · Paper page
- Patro, B., Patel, S., and Namboodiri, V. (2020, January 1–5). Robust explanations for visual question answering. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA. · DOI
- Wu, Q., Wang, P., Shen, C., Reid, I., and Van Den Hengel, A. (2018, January 18–22). Are you talking to me? reasoned visual dialog generation through adversarial learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. · DOI
- Chen, C., Mu, S., Xiao, W., Ye, Z., Wu, L., and Ju, Q. (February, January 27). Improving image captioning with conditional generative adversarial nets. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
- Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015;, January 7–9). Show, attend and tell: Neural image caption generation with visual attention. Proceedings of the International Conference on Machine Learning, Lille, France.
- Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
- Mnih, 2014, Recurrent models of visual attention, Adv. Neural Inf. Process. Syst., 27, 2204
- Lu, J., Xiong, C., Parikh, D., and Socher, R. (2017, January 21–26). Knowing when to look: Adaptive attention via a visual sentinel for image captioning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA. · DOI · Paper page
- Qin, Y., Du, J., Zhang, Y., and Lu, H. (2019, January 16–17). Look back and predict forward in image captioning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA. · DOI
- Ke, L., Pei, W., Li, R., Shen, X., and Tai, Y.W. (November, January 27). Reflective decoding network for image captioning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Repulic of Korea.
- Huang, L., Wang, W., Chen, J., and Wei, X.Y. (November, January 27). Attention on attention for image captioning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Repulic of Korea.
- Guo, L., Liu, J., Zhu, X., Yao, P., Lu, S., and Lu, H. (2020, January 14–19). Normalized and geometry-aware self-attention network for image captioning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA. · DOI · Paper page
- Li, G., Zhu, L., Liu, P., and Yang, Y. (November, January 27). Entangled transformer for image captioning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Repulic of Korea.
- Yao, T., Pan, Y., Li, Y., and Mei, T. (2018, January 8–14). Exploring visual relationship for image captioning. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany. · DOI · Paper page
- Yang, X., Tang, K., Zhang, H., and Cai, J. (2019, January 16–17). Auto-encoding scene graphs for image captioning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA. · DOI · Paper page
- Yao, T., Pan, Y., Li, Y., and Mei, T. (November, January 27). Hierarchy parsing for image captioning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Repulic of Korea.
- Aneja, J., Deshpande, A., and Schwing, A.G. (2018, January 18–22). Convolutional image captioning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. · DOI · Paper page
- Wang, Q., and Chan, A.B. (2018). Cnn+ cnn: Convolutional decoders for image captioning. arXiv.
- Herdade, 2019, Image captioning: Transforming objects into words, Adv. Neural Inf. Process. Syst., 32, 11135
- Luo, Y., Ji, J., Sun, X., Cao, L., Wu, Y., Huang, F., Lin, C.W., and Ji, R. (2021). Dual-level collaborative transformer for image captioning. arXiv. · DOI · Paper page
- Johnson, J., Karpathy, A., and Fei-Fei, L. (2016, January 27–30). Densecap: Fully convolutional localization networks for dense captioning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. · DOI · Paper page
- Yang, L., Tang, K., Yang, J., and Li, L.J. (2017, January 21–26). Dense captioning with joint inference and visual context. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA. · DOI · Paper page
- Kim, D.J., Choi, J., Oh, T.H., and Kweon, I.S. (2019, January 16–17). Dense relational captioning: Triple-stream networks for relationship-based captioning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA. · DOI
- Uehara, K., Mori, Y., Mukuta, Y., and Harada, T. (2022, January 25–29). ViNTER: Image Narrative Generation with Emotion-Arc-Aware Transformer. Proceedings of the Companion Proceedings of the Web Conference 2022, Virtual Event/Lyon, France. · DOI
- Su, 2021, BERT-hLSTMs: BERT and hierarchical LSTMs for visual storytelling, Comput. Speech Lang., 67, 101169, 10.1016/j.csl.2020.101169 · DOI
- 2020, Image Collection Summarization Method Based on Semantic Hierarchies, AI, 1, 209, 10.3390/ai1020014 · DOI
- Sharma, V., Kumar, A., Agrawal, N., Singh, P., and Kulshreshtha, R. (2015, January 19–21). Image summarization using topic modelling. Proceedings of the 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia. · DOI
- Camargo, J.E., and González, F.A. (2009, January 15–18). A multi-class kernel alignment method for image collection summarization. Proceedings of the Iberoamerican Congress on Pattern Recognition, Guadalajara, Mexico. · doi:10.1007/978-3-642-10268-4_64
- Sreelakshmi, 2021, Image Summarization Using Unsupervised Learning, Proceedings of the 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Volume 1, 100
- Chen, 2019, Extractive summarization of documents with images based on multi-modal RNN, Future Gener. Comput. Syst., 99, 186, 10.1016/j.future.2019.04.045 · DOI
- Qian, 2019, Social media based event summarization by user–text–image co-clustering, Knowl. Based Syst., 164, 107, 10.1016/j.knosys.2018.10.028 · doi:10.1016/j.knosys.2018.10.028
- Kuzovkin, D., Pouli, T., Cozot, R., Meur, O.L., Kervec, J., and Bouatouch, K. (2017, January 29–30). Context-aware clustering and assessment of photo collections. Proceedings of the Symposium on Computational Aesthetics, Los Angeles, CA, USA. · DOI
- Camargo, J.E., and González, F.A. (2011, January 4–6). Multimodal image collection summarization using non-negative matrix factorization. Proceedings of the 2011 6th Colombian Computing Congress (CCC), Manizales, Colombia. · doi:10.1109/colomcc.2011.5936291
- Huang, T.H., Ferraro, F., Mostafazadeh, N., Misra, I., Agrawal, A., Devlin, J., Girshick, R., He, X., Kohli, P., and Batra, D. (2016, January 12–17). Visual storytelling. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA. · DOI · Paper page
- Yu, L., Bansal, M., and Berg, T.L. (2017). Hierarchically-attentive rnn for album summarization and storytelling. arXiv. · DOI
- Wang, X., Chen, W., Wang, Y.F., and Wang, W.Y. (2018). No metrics are perfect: Adversarial reward learning for visual storytelling. arXiv. · DOI · Paper page
- Wang, J., Fu, J., Tang, J., Li, Z., and Mei, T. (2018, January 2–7). Show, reward and tell: Automatic generation of narrative paragraph from photo stream by adversarial training. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA. · DOI
- Huang, Q., Gan, Z., Celikyilmaz, A., Wu, D., Wang, J., and He, X. (February, January 27). Hierarchically structured reinforcement learning for topically coherent visual story generation. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
- Hsu, C.C., Chen, Z.Y., Hsu, C.Y., Li, C.C., Lin, T.Y., Huang, T.H., and Ku, L.W. (2020, January 7–12). Knowledge-enriched visual storytelling. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
- Jung, Y., Kim, D., Woo, S., Kim, K., Kim, S., and Kweon, I.S. (2020, January 7–12). Hide-and-tell: Learning to bridge photo streams for visual storytelling. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA. · DOI
- Hu, J., Cheng, Y., Gan, Z., Liu, J., Gao, J., and Neubig, G. (2020, January 7–12). What makes a good story? Designing composite rewards for visual storytelling. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA. · DOI
- Yu, Y., Chung, J., Yun, H., Kim, J., and Kim, G. (2021, January 19–25). Transitional Adaptation of Pretrained Models for Visual Storytelling. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA. · DOI
- Lukin, S.M., Hobbs, R., and Voss, C.R. (2018). A pipeline for creative visual storytelling. arXiv. · DOI
- Xu, C., Yang, M., Li, C., Shen, Y., Ao, X., and Xu, R. (2021, January 2–9). Imagine, Reason and Write: Visual Storytelling with Graph Knowledge and Relational Reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual. · DOI
- Wang, E., Han, C., and Poon, J. (2022). RoViST: Learning Robust Metrics for Visual Storytelling. arXiv. · DOI
- Li, T., Wang, H., He, B., and Chen, C.W. (IEEE Trans. Pattern Anal. Mach. Intell., 2022). Knowledge-enriched attention network with group-wise semantic for visual storytelling, IEEE Trans. Pattern Anal. Mach. Intell., Early Access. · DOI
- Knaflic, C.N. (2019). Storytelling with Data: Let’s Practice!, John Wiley & Sons.
- Toker, D., Conati, C., Carenini, G., and Haraty, M. (2012, January 16–20). Towards adaptive information visualization: On the influence of user characteristics. Proceedings of the International Conference on User Modeling, Adaptation, and Personalization, Montreal, QC, Canada. · DOI
- Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 7–12). Bleu: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA. · DOI · Paper page
- Banerjee, S., and Lavie, A. (2005, January 29). METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, MI, USA.
- Anderson, P., Fernando, B., Johnson, M., and Gould, S. (2016, January 11–14). Spice: Semantic propositional image caption evaluation. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands. · DOI · Paper page
- Vedantam, R., Lawrence Zitnick, C., and Parikh, D. (2015, January 7–12). Cider: Consensus-based image description evaluation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
- Yao, L., Peng, N., Weischedel, R., Knight, K., Zhao, D., and Yan, R. (February, January 27). Plan-and-write: Towards better automatic storytelling. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
- Chen, Y.C., and Bansal, M. (2018). Fast abstractive summarization with reinforce-selected sentence rewriting. arXiv. · DOI · Paper page
- Aafaq, 2019, Video description: A survey of methods, datasets, and evaluation metrics, ACM Comput. Surv. (CSUR), 52, 1, 10.1145/3355390 · DOI · Paper page
- Kilickaya, M., Erdem, A., Ikizler-Cinbis, N., and Erdem, E. (2016). Re-evaluating automatic metrics for image captioning. arXiv. · DOI
- Sai, 2022, A survey of evaluation metrics used for NLG systems, ACM Comput. Surv. (CSUR), 55, 1, 10.1145/3485766 · DOI · Paper page
- Engel, 2013, Breaking with trends in pre-processing?, TrAC Trends Anal. Chem., 50, 96, 10.1016/j.trac.2013.04.015 · DOI · Paper page
- Hemanth, D.J., and Anitha, J. (2012, January 12–14). Image pre-processing and feature extraction techniques for magnetic resonance brain image analysis. Proceedings of the International Conference on Future Generation Communication and Networking, London, UK.
- Rajesh, S.D., Almeida, J.M., and Martins, A. (2019, January 17–20). Image Cleaning and Enhancement Technique for Underwater Mining. Proceedings of the OCEANS 2019, Marseille, France. · doi:10.1109/oceanse.2019.8866882
- Jiang, Z., Zhang, Y., Zou, D., Ren, J., Lv, J., and Liu, Y. (2020, January 14–19). Learning event-based motion deblurring. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA. · DOI · Paper page
- Uijlings, 2013, Selective search for object recognition, Int. J. Comput. Vis., 104, 154, 10.1007/s11263-013-0620-5 · DOI · Paper page
- Lazebnik, 2006, Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), Volume 2, 2169, 10.1109/CVPR.2006.68 · DOI · Paper page
- Shin, A., Ushiku, Y., and Harada, T. (2018, January 18–22). Customized Image Narrative Generation via Interactive Visual Question Generation and Answering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. · DOI
- Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
- Fang, H., Gupta, S., Iandola, F., Srivastava, R.K., Deng, L., Dollár, P., Gao, J., He, X., Mitchell, M., and Platt, J.C. (2015, January 7–12). From captions to visual concepts and back. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
- Wu, Q., Shen, C., Liu, L., Dick, A., and Van Den Hengel, A. (2016, January 27–30). What value do explicit high level concepts have in vision to language problems?. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. · DOI · Paper page
- Li, Y., Duan, N., Zhou, B., Chu, X., Ouyang, W., Wang, X., and Zhou, M. (2018, January 18–22). Visual question generation as dual task of visual question answering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. · DOI · Paper page
- Kipf, T.N., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv.
- Malakan, Z.M., Hassan, G.M., and Mian, A. (2022, January 5–8). Vision Transformer Based Model for Describing a Set of Images as a Story. Proceedings of the AI 2022: Advances in Artificial Intelligence: 35th Australasian Joint Conference, AI 2022, Perth, WA, Australia. · DOI
- Cao, 2022, Vision-Enhanced and Consensus-Aware Transformer for Image Captioning, IEEE Trans. Circuits Syst. Video Technol., 32, 7005, 10.1109/TCSVT.2022.3178844 · DOI
- Fang, Z., Wang, J., Hu, X., Liang, L., Gan, Z., Wang, L., Yang, Y., and Liu, Z. (2022, January 18–24). Injecting semantic concepts into end-to-end image captioning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA. · DOI
- Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
- Vinyals, 2016, Show and tell: Lessons learned from the 2015 mscoco image captioning challenge, IEEE Trans. Pattern Anal. Mach. Intell., 39, 652, 10.1109/TPAMI.2016.2587640 · DOI · Paper page
- Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., and Darrell, T. (2015, January 7–12). Long-term recurrent convolutional networks for visual recognition and description. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
- Gu, J., Wang, G., Cai, J., and Chen, T. (2017, January 22–29). An empirical study of language cnn for image captioning. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy. · DOI
- Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
- Zhu, Z., Wei, Y., Wang, J., Gan, Z., Zhang, Z., Wang, L., Hua, G., Wang, L., Liu, Z., and Hu, H. (2022). Exploring Discrete Diffusion Models for Image Captioning. arXiv.
- Luo, J., Li, Y., Pan, Y., Yao, T., Feng, J., Chao, H., and Mei, T. (2022). Semantic-Conditional Diffusion Networks for Image Captioning. arXiv.
- Xu, S. (2022). CLIP-Diffusion-LM: Apply Diffusion Model on Image Captioning. arXiv.
- Cheung, J.C.K., and Li, X. (2012, January 8–12). Sequence clustering and labeling for unsupervised query intent discovery. Proceedings of the Fifth ACM International Conference on Web Search and Data Mining, Seattle, WA, USA. · DOI
- Vedula, N., Lipka, N., Maneriker, P., and Parthasarathy, S. (2019). Towards open intent discovery for conversational text. arXiv.
- Narayanan, A., Chandramohan, M., Venkatesan, R., Chen, L., Liu, Y., and Jaiswal, S. (2017). graph2vec: Learning distributed representations of graphs. arXiv.
- Beheshti, A., Benatallah, B., Nouri, R., Chhieng, V.M., Xiong, H., and Zhao, X. (2017, January 6–10). Coredb: A data lake service. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore. · doi:10.1145/3132847.3133171
- Beheshti, 2018, CoreKG: A knowledge lake service, Proc. VLDB Endow., 11, 1942, 10.14778/3229863.3236230 · doi:10.14778/3229863.3236230
- Li, N., Liu, B., Han, Z., Liu, Y.S., and Fu, J. (2019, January 10–13). Emotion reinforced visual storytelling. Proceedings of the 2019 on International Conference on Multimedia Retrieval, Ottawa, ON, Canada. · DOI
- Hsu, C.Y., Chu, Y.W., Huang, T.H., and Ku, L.W. (2021). Plot and Rework: Modeling Storylines for Visual Storytelling. arXiv. · DOI
- Nahian, M., Al, S., Tasrin, T., Gandhi, S., Gaines, R., and Harrison, B. (2019, January 19–22). A hierarchical approach for visual storytelling using image description. Proceedings of the International Conference on Interactive Digital Storytelling, Little Cottonwood Canyon, UT, USA. · DOI
- Kim, T., Heo, M.O., Son, S., Park, K.W., and Zhang, B.T. (2018). Glac net: Glocal attention cascading networks for multi-image cued story generation. arXiv.
- Wang, R., Wei, Z., Li, P., Zhang, Q., and Huang, X. (2020, January 7–12). Storytelling from an image stream using scene graphs. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
- Zhang, B., Hu, H., and Sha, F. (2020). Visual storytelling via predicting anchor word embeddings in the stories. arXiv.
- Gonzalez-Rico, D., and Fuentes-Pineda, G. (2018). Contextualize, show and tell: A neural visual storyteller. arXiv.
- Wang, P., Zamora, J., Liu, J., Ilievski, F., Chen, M., and Ren, X. (2021). Contextualized scene imagination for generative commonsense reasoning. arXiv.
- Smilevski, M., Lalkovski, I., and Madjarov, G. (2018, January 26–28). Stories for images-in-sequence by using visual and narrative components. Proceedings of the International Conference on Telecommunications, Saint Malo, France. · DOI
- Chandu, K., Nyberg, E., and Black, A.W. (August, January 28). Storyboarding of recipes: Grounded contextual generation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
- Salvador, A., Gundogdu, E., Bazzani, L., and Donoser, M. (2021, January 19–25). Revamping cross-modal recipe retrieval with hierarchical transformers and self-supervised learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA. · DOI
- Nishimura, 2020, Structure-aware procedural text generation from an image sequence, IEEE Access, 9, 2125, 10.1109/ACCESS.2020.3043452 · DOI
- Qi, M., Qin, J., Huang, D., Shen, Z., Yang, Y., and Luo, J. (2021, January 20–24). Latent Memory-augmented Graph Transformer for Visual Storytelling. Proceedings of the 29th ACM International Conference on Multimedia, Virtual. · DOI
- Hong, X., Shetty, R., Sayeed, A., Mehra, K., Demberg, V., and Schiele, B. (2020, January 19–20). Diverse and Relevant Visual Storytelling with Scene Graph Embeddings. Proceedings of the 24th Conference on Computational Natural Language Learning, Online. · DOI
- Joana, 2021, Quality assessment for digital stories by young authors, Data Inf. Manag., 5, 174
- Kusner, M., Sun, Y., Kolkin, N., and Weinberger, K. (2015, January 7–9). From word embeddings to document distances. Proceedings of the International Conference on Machine Learning, Lille, France.
- Lin, C.Y. (2004). Text Summarization Branches Out, Association for Computational Linguistics.
- Hsu, T.Y., Huang, C.Y., Hsu, Y.C., and Huang, T.H. (2019). Visual story post-editing. arXiv. · DOI
- Ravi, H., Kafle, K., Cohen, S., Brandt, J., and Kapadia, M. (2021, January 10–17). AESOP: Abstract Encoding of Stories, Objects, and Pictures. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada. · DOI
- Hsu, C.Y., Chu, Y.W., Chen, V., Lo, K.C., Chen, C., Huang, T.H., and Ku, L.W. (2022, January 22–27). Learning to Rank Visual Stories From Human Ranking Data. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland. · DOI
- Marin, 2019, Recipe1m+: A dataset for learning cross-modal embeddings for cooking recipes and food images, IEEE Trans. Pattern Anal. Mach. Intell., 43, 187, 10.1109/TPAMI.2019.2927476 · DOI · Paper page
- Wang, H., Lin, G., Hoi, S.C., and Miao, C. (2020). Decomposed generation networks with structure prediction for recipe generation from food images. arXiv.
- Nishimura, T., Hashimoto, A., Ushiku, Y., Kameko, H., and Mori, S. (2022). Recipe Generation from Unsegmented Cooking Videos. arXiv.
- Fain, M., Twomey, N., Ponikar, A., Fox, R., and Bollegala, D. (2019). Dividing and conquering cross-modal recipe retrieval: From nearest neighbours baselines to sota. arXiv.
- Sakib, 2022, Approximate task tree retrieval in a knowledge network for robotic cooking, IEEE Robot. Autom. Lett., 7, 11492, 10.1109/LRA.2022.3191068 · DOI
Cited by
Designing for Reminiscence in Digital Photo Systems for Older Adults: A Scoping Review
Shen, Y. (2026) · AHFE International · doi:10.54941/ahfe1007577
Metadata
- Authors
- Lotfi, F., Beheshti, A., Farhood, H., Pooshideh, M., Jamzad, M., & Beigy, H.
- Affiliations
- School of Computing, Macquarie University, Sydney 2109, Australia
- Computer Engineering Department, Sharif University of Technology, Tehran 14588-89694, Iran
- Venue
- Algorithms
- Publisher
- MDPI AG
- Published
- 2023-03-02
- Volume
- 16
- Issue
- 3
- Pages
- 135
- ISSN
- 1999-4893
- Cited by
- 24
- References
- 17
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Storytelling with Image Data: A Systematic Review and Comparative Analysis of Methods and Tools. https://app.cochat.ai/papers/10.3390/a16030135
- The paper
- Lotfi, F., Beheshti, A., Farhood, H., Pooshideh, M., Jamzad, M., & Beigy, H. (2023). Storytelling with Image Data: A Systematic Review and Comparative Analysis of Methods and Tools. Algorithms. https://doi.org/10.3390/a16030135
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.