The DOI matches its Crossref record; 280 of the 280 references it cites were found in a registry or an index; none of the 144 sources its claims cite is checked for support yet.
A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets
Open accessAbstract
The research progress in multimodal learning has grown rapidly over the last decade in several areas, especially in computer vision. The growing potential of multimodal data streams and deep learning algorithms has contributed to the increasing universality of deep multimodal learning. This involves the development of models capable of processing and analyzing the multimodal information uniformly. Unstructured real-world data can inherently take many forms, also known as modalities, often including visual and textual content. Extracting relevant patterns from this kind of data is still a motivating goal for researchers in deep learning. In this paper, we seek to improve the understanding of key concepts and algorithms of deep multimodal learning for the computer vision community by exploring how to generate deep models that consider the integration and combination of heterogeneous visual cues across sensory modalities. In particular, we summarize six perspectives from the current literature on deep multimodal learning, namely: multimodal data representation, multimodal fusion (i.e., both traditional and deep learning-based schemes), multitask learning, multimodal alignment, multimodal transfer learning, and zero-shot learning. We also survey current multimodal applications and present a collection of benchmark datasets for solving problems in various vision domains. Finally, we highlight the limitations and challenges of deep multimodal learning and provide insights and directions for future research.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This survey reviews deep multimodal learning for computer vision, organizing advances into six perspectives: multimodal data representation, multimodal fusion (traditional and deep learning-based), multitask learning, multimodal alignment, multimodal transfer learning, and zero-shot learning. It also surveys applications and benchmark datasets and highlights limitations and challenges including dimensionality/data conflict, data availability, and real-time processing/scalability.
Key findings
- The survey organizes deep multimodal learning into six topics: multimodal data representation, multimodal fusion (traditional and deep learning-based schemes), multitask learning, multimodal alignment, multimodal transfer learning, and zero-shot learning.
- It reviews conventional fusion methods (early, late, hybrid, kernel-based, graphical models, canonical correlation analysis) and deep fusion methods based on DBNs, DAEs, CNNs, RNNs, GANs, and attention mechanisms, noting model-specific tradeoffs such as DBNs largely ignoring spatiotemporal cues, DAEs missing spatiotemporal details, CNNs requiring large-scale multimodal datasets, RNNs having slow convergence, and GANs/attention being unstable to train.
- It surveys multimodal applications including object detection, visual tracking, semantic segmentation, human/face recognition, image retrieval, gesture recognition, image captioning, vision-and-language navigation, embodied question answering, video question answering, style transfer, medical data analysis, and autonomous systems. Table 2 reports prior best results such as face recognition on LFW (ACC = 98.43%) and CASIA-WebFace (ACC = 99.02%), gesture recognition on SKIG (ACC = 97.8%), video question answering on TDIUC (ACC = 88.20%), and autonomous systems on nuScenes (mAP = 28.9%; NDS = 44.9%).
- It presents benchmark datasets including RGB-D Object (300 object instances, 51 categories, 250,000 RGB-D images), BigBIRD (125 objects, 600 RGB-D point clouds, 600 12-megapixel images), Kinetics-400 (more than 300,000 video sequences across 400 classes), SceneNet RGB-D (5M RGB-D images), and nuScenes (1000 scenes, each about 20 s long, 23 classes, eight attributes).
Methods
- Survey/overview paper; no new experiments are reported. It reviews and synthesizes literature, compares its scope with previous surveys in Table 1, and organizes the field into six perspectives.
- It uses tables to summarize reviewed deep multimodal learning surveys, multimodal applications with technical details and best results (Table 2), and frequently used multimodal datasets (Tables 3 and 4), focusing on visual modalities such as RGB-D and RGB-optical flow.
- The main analysis is qualitative synthesis and taxonomy; no primary data collection, statistical meta-analysis, or systematic review protocol is stated.
Limitations
- The paper states main challenges of deep multimodal learning: dimensionality and data conflict, data availability, and real-time processing and scalability. It also states that despite impressive results, “no absolute conclusions can be drawn.”
- It lists model-specific limitations: DBNs largely ignore spatiotemporal cues and have computationally intensive inference; DAEs miss some spatiotemporal details and have many hidden parameters with time-consuming inference; CNNs require large-scale multimodal datasets and time-consuming inference; RNNs have slow convergence; GANs are unstable and tricky to train on discrete distributions; attention mechanisms have unstable training and a huge number of parameters.
- Evident from the text: as a survey, it does not provide new empirical validation or a quantitative meta-analysis beyond reporting prior results, and it does not state a systematic review protocol or inclusion criteria.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 144 cited sources checked · 0 supported · 0 flagged
In the paper
“Another crucial advantage of inter- and intra-modal interaction is the discriminating power of the perceptual model for multisensory stimuli by exploiting the potential synergies between modalities and their intrinsic representations [112].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal representation
What the sources say
- Not checked yetcites [112] Peng, Y., et al.: CCL: cross-modal correlation learning with multigrained fusion by hierarchical network. IEEE Trans. Multimed. 20(2), 405–420 (2017) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“but to the best of our knowledge, there is no recent work that directly addresses the latest advances in deep multimodal learning particularly for the computer vision community.”
- Also stated in: § Comparison with previous surveys · hedged
“However, there is still relatively little recent work directly addressing this research area [32–37].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Comparison with previous surveys
What the sources say
- Not checked yetcites [32] Marechal, C. et al.: Survey on AI-based multimodal methods for emotion detection. In: High-Performance Modelling and Simulation for Big Data Applications: Selected Results of the COST Action IC1406 cHiPSet, pp. 307–324 (2019) source
- Not checked yetcites [33] Radu, V., et al.: Multimodal deep learning for activity and context recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 157:1–157:27 (2018) source
- Not checked yetcites [34] Ramachandram, D., Taylor, G.W.: Deep multimodal learning: a survey on recent advances and trends. IEEE Signal Process. Mag. 34, 96–108 (2017) source
- Not checked yetcites [35] Guo, W., Wang, J., Wang, S.: Deep multimodal representation learning: a survey. IEEE Access 7, 63373–63394 (2019) source
- Not checked yetcites [36] Lahat, D., Adali, T., Jutten, C.: Multimodal data fusion: an overview of methods, challenges, and prospects. Proc. IEEE 103(9), 1449–1477 (2015) source
- Not checked yetcites [37] Baltrušaitis, T., Ahuja, C., Morency, L.-P.: Multimodal Machine Learning: A Survey and Taxonomy. arXiv:1705.09406 (2017)
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: A majority of contemporary machine learning applications use multiple modalities, such as visual and textual data.
- Also stated in: § Comparison with previous surveys · hedged
None of the sources cited for this claim have been checked yet.
In the paper
“Recently, the valuable works of [32, 33] considered several multimodal practices that apply only to specific multimodal use cases and applications, such as emotion recognition [32], human activity and context recognition [33].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Comparison with previous surveys
What the sources say
- Not checked yetcites [32] Marechal, C. et al.: Survey on AI-based multimodal methods for emotion detection. In: High-Performance Modelling and Simulation for Big Data Applications: Selected Results of the COST Action IC1406 cHiPSet, pp. 307–324 (2019) source
- Not checked yetcites [33] Radu, V., et al.: Multimodal deep learning for activity and context recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 157:1–157:27 (2018) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Nonetheless, the applications discussed in these surveys include only a single modality as a data source for data-driven learning.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Comparison with previous surveys
What the sources say
- Not checked yetcites [4] LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015) source
- Not checked yetcites [180] Hatcher, W.G., Yu, W.: A survey of deep learning: platforms, applications and emerging research trends. IEEE Access 6, 24411–24432 (2018) source
- Not checked yetcites [182] Pouyanfar, S., et al.: A survey on deep learning: algorithms, techniques, and applications. ACM Comput. Surv. 51, 92:1–92:36 (2018)
- Not checked yetcites [179] Guo, Y., et al.: Deep learning for visual understanding: a review. Neurocomputing 187, 27–48 (2016) source
- Not checked yetcites [181] Wu, X., Sahoo, D. Hoi, S.C.H.: Recent Advances in Deep Learning for Object Detection. arXiv:1908.03673 (2019) source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Furthermore, some cutting-edge works [34, 36] have been proposed in recent years that address the mechanism of integrating and fusing multimodal representations inside deep learning architectures by showing the reader the possibilities this opens up for the artificial intelligence community.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Comparison with previous surveys
What the sources say
- Not checked yetcites [34] Ramachandram, D., Taylor, G.W.: Deep multimodal learning: a survey on recent advances and trends. IEEE Signal Process. Mag. 34, 96–108 (2017) source
- Not checked yetcites [36] Lahat, D., Adali, T., Jutten, C.: Multimodal data fusion: an overview of methods, challenges, and prospects. Proc. IEEE 103(9), 1449–1477 (2015) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“In addition to their use for unimodal tasks, RNNs have proved useful in many multimodal problems that require modeling long- and short-range dependencies across the input sequence, such as semantic segmentation [219] and image captioning [220].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [219] Abdulnabi, A.H., et al.: Multimodal recurrent neural networks with information transfer layers for indoor scene labeling. IEEE Trans. Multimed. 20, 1656–1671 (2018) source
- Not checked yetcites [220] Zhao, D., et al.: A multimodal fusion approach for image captioning. Neurocomputing 329, 476–485 (2019) source
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Since its introduction by Krizhevsky et al. [7] in 2012, the CNN model has been successfully applied to a wide range of multimodal applications, such as image dehazing [239, 240] and human activity recognition [241].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [7] Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84–90 (2017) source
- Not checked yetcites [239] Zhang, S., et al.: Joint learning of image detail and transmission map for single image dehazing. Vis. Comput. 36, 305–316 (2020) source
- Not checked yetcites [240] Zhang, S., He, F.: DRCDN: learning deep residual convolutional dehazing networks. Vis. Comput. 36, 1797–1808 (2020) source
- Not checked yetcites [241] Basly, H., et al.: DTR-HAR: deep temporal residual representation for human activity recognition. Vis. Comput. (2021). https://doi.org/10.1007/s00371-021-02064-y source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Furthermore, an AM can be integrated into RNN models to improve the generalization capability of the former by capturing the most representative and discriminating patterns from heterogeneous datasets.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Since its introduction, the AM has gained wide adoption in the computer vision community due to its spectral capabilities for many multimodal applications such as video description [233, 234], salient object detection [235], etc.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [233] Hori, C., et al.: Attention-based multimodal fusion for video description. In: IEEE International Conference on Computer Vision (ICCV), pp. 4203–4212 (2017) source
- Not checked yetcites [234] Huang, X., Wang, M., Gong, M.: Fine-grained talking face generation with video reinterpretation. Vis. Comput. 37, 95–105 (2021) source
- Not checked yetcites [235] Liu, Z., et al.: Multi-level progressive parallel attention guided salient object detection for RGB-D images. Vis. Comput. (2020). https://doi.org/10.1007/s00371-020-01821-9 source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIn the paper
“An adaptive multimodal mapping between two visual modalities (e.g., images and sentences) typically requires strong representations of the individual modalities [213].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [213] Ma, L., et al.: Multimodal convolutional neural networks for matching image and sentence. In: IEEE International Conference on Computer Vision (ICCV), pp. 2623–2631 (2015) source
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
- Probably not a substantial claim. 120
- It may state two separate claims as one. 28
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As shown in Fig. 3, the textual representation (i.e., a word embedding) is very sparse when compared to the image one, which makes it very challenging to combine these two different representations into a unified model.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal representation
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Figure 6 shows the coupling mechanism of two separate AEs (bimodal AE) for both modalities (audio and video) into a jointly shared representation hierarchy where the encoder and decoder components are independent of each other.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As a powerful tool for feature extraction and dimensionality reduction, the DAE aims to learn how to efficiently represent manifolds where the training data is unbalanced or lacking.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, they also miss some spatiotemporal details in multimodal data.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Guo et al. [35] provided a comprehensive overview of deep multimodal learning frameworks and models, focusing on one of the main challenges of multimodal learning, namely multimodal representation.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Comparison with previous surveys
What the sources say
- Not checked yetcites Guo et al. [35] Guo, W., Wang, J., Wang, S.: Deep multimodal representation learning: a survey. IEEE Access 7, 63373–63394 (2019) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Here, the authors identified five levels of multimodal data combination: representation, translation, alignment, fusion, and co-learning.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Comparison with previous surveys
What the sources say
- Not checked yetcites Baltrušaitis et al. [37] Baltrušaitis, T., Ahuja, C., Morency, L.-P.: Multimodal Machine Learning: A Survey and Taxonomy. arXiv:1705.09406 (2017)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One of the main drawbacks of DAEs is that many hidden parameters have to be trained, and the inference process is time-consuming.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“the authors of [210–212] designed multimodal systems based on SAEs, where the encoder side of the architecture represents and compresses each unimodal feature separately, and the decoder side constructs the latent (shared) representation of the inputs in a unsupervised manner.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [210] Kim, J., Chung, K.: Multi-modal stacked denoising autoencoder for handling missing data in healthcare big data. IEEE Access 8, 104933–104943 (2020) source
- Not checked yetcites [211] Singh, V., et al.: Feature learning using stacked autoencoder for shared and multimodal fusion of medical images. In: Computational Intelligence: Theories, Applications and Future Directions, pp. 53–66 (2019) source
- Not checked yetcites [212] Said, A. B., et al.: Multimodal deep learning approach for joint EEG-EMG data compression and classification. In: IEEE Wireless Communications and Networking Conference (WCNC), pp. 1–6 (2017) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, most of the papers we reviewed are recent and have been published in high-quality conferences and journals such as the visual computer, ICCV, and CVPR.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Comparison with previous surveys
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The research progress in multimodal learning has grown rapidly over the last decade in several areas, especially in computer vision.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Abstract
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Therefore, the AM acts as a contextual bridge between the encoding and decoding sides of an RNN to pay attention only to a particular position in the input representation.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The idea of AM is to focus on a particular position in the input image by computing the weighted sum of feature vectors and mapping them into a final contextual representation.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Structurally, GAN is a unified network consisting of two sub-networks, a generator network (G) and a discriminator network (D), which interact continuously during the learning process.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“What they have in common is that the sharing of the structure between all tasks can be learned jointly to improve performance.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multitask learning
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the only weakness of AM is that the training algorithm is unstable, which may affect the predictive power of the decision-making system.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“More recently, multitask learning (MTL) [108, 109] has become an increasingly popular topic in the deep learning community.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multitask learning
What the sources say
- Not checked yetcites [108] Ruder, S.: An Overview of Multi-Task Learning in Deep Neural Networks. arXiv:1706.05098 (2017)
- Not checked yetcites [109] Ruder, S., Bingel, J., Augenstein, I., Søgaard, A.: Latent Multi-task Architecture Learning. arXiv:1705.08142 (2018) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, it requires a large-scale multimodal dataset to converge optimally during training, and the inference process is time-consuming.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In other words, it learns how to reduce some irrelevant attributes from a set of feature vectors.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Typically, MTL contributes to the performance of the target task based on knowledge gained from auxiliary tasks.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multitask learning
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Generative adversarial networks (GANs) are part of deep generative architectures, designed to learn the data distribution through the adversarial learning.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, it requires a large-scale multimodal dataset to converge optimally during training, and the inference process is time-consuming.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, the function that this network seeks to optimize is an adversarial loss function without any normalization.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, it is extremely tricky to train the network when generating discrete distributions, and the process itself is unstable compared to other generative networks.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the model has a slow convergence rate due to the high computational cost in the hidden state transfer function.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Hard parameter sharing [110]: It consists of extracting a generic representation for different tasks using the same parameters. It is usually applied to avoid overfitting problems.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multitask learning
What the sources say
- Not checked yetcites [110] Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, the multimodal AM provides a robust solution for cross-modal data fusion by selecting the local fine-grained salient features in a multidimensional space and filtering out any hidden noise.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Generally, the alignment task can be divided into two subtasks: the attentional alignment task [170, 171] and the semantic alignment task [172, 173].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal alignment
What the sources say
- Not checked yetcites [170] Dapogny, A., et al.: Deep Entwined Learning Head Pose and Face Alignment Inside an Attentional Cascade with Doubly-Conditional fusion. arXiv:2004.06558 (2020) source
- Not checked yetcites [171] Yue, L., et al.: Attentional alignment network. In: BMVC (2018)
- Not checked yetcites [172] Liu, Z., et al.: Semantic Alignment: Finding Semantically Consistent Ground-truth for Facial Landmark Detection. arXiv:1903.10661 (2019) source
- Not checked yetcites [173] Hao, F., et al.: Collect and select: semantic alignment metric learning for few-shot learning. In: CVPR, pp. 8460–8469 (2019) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“More specifically, such models tend to model a series of nonlinear transformations by generating very abstract and informative features from highly complex datasets.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In terms of computational efficiency, GRU is a lightweight variant of LSTM since it can modulate the information flow without using its internal memory units.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the multimodal analysis, an attentional model can be designed to combine multiple modalities, each with its internal representation (e.g., spatial features, motion features, etc.).”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Multimodal alignment consists of linearly linking the features of two or more different modalities.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal alignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Patch-based methods first split each image into local patches and then consider the similarity between them by computing their cross-correlation and combination.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal alignment
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Recurrent neural networks (RNNs) [12] are a popular type of deep neural network architectures for processing sequential data of varying lengths.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [12] Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In recent years, the attention mechanism (AM) has become one of the most challenging tasks in computer vision and machine translation [232].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [232] Chaudhari, S., et al.: An Attentive Survey of Attention Models. arXiv:1904.02874 (2020)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, the vanilla RNN model is typically incapable of capturing long-term dependencies in sequential data since they have no internal memory.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In particular, CNNs have demonstrated powerful generalization capabilities to learn how to represent visual appearance features from static data.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, unsupervised GAN is one of the most powerful generative models that can address scenarios where training data is lacking or some hidden concepts are missing.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Since their development in 2014, generative adversarial training algorithms have been widely used in various unimodal applications such as scene generation [17], image-to-image translation [18], and image super-resolution [224, 225].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [17] Turkoglu, M.O., Thong, W., Spreeuwers, L., Kicanaoglu, B.: A layer-based sequential framework for scene generation with GANs. arXiv:1902.00671 (2019) source
- Not checked yetcites [18] Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. arXiv:1611.07004 (2018) source
- Not checked yetcites [224] Song, H., et al.: $${\rm S}^2{{\rm RGAN}}$$S: sonar-image super-resolution based on generative adversarial network. Vis. Comput. (2020). https://doi.org/10.1007/s00371-020-01986-3 source
- Not checked yetcites [225] Ma, T., Tian, W.: Back-projection-based progressive growing generative adversarial network for single image super-resolution. Vis. Comput. (2020). https://doi.org/10.1007/s00371-020-01843-3 source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It also can handle time series and dynamic media such as text and video sequences.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Specifically, the MTL paradigm frequently arises in a context close to multimodal concepts.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multitask learning
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is also capable of modeling spatial cues from multimodal data streams with an increased number of parameters.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Feature-based methods detect and extract a set of matching features that should be structurally consistent to describe their spatial patterns.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal alignment
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“They learn to map input activations to the next hierarchy level and then transfer hidden states to the outputs using the recurrent feedback, which gives them the capacity to learn useful features from the previous states, unlike other deep feedforward networks such as CNNs, DBNs, etc.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, the number of parameters to be trained is huge compared to other deep networks such as RNNs, CNNs, etc.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Specifically, multimodal image alignment provides a spatial mapping capability between images taken by sensors of different modalities, which may be categorized into feature-based [167, 168] and patch-based [165, 166] methods.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal alignment
What the sources say
- Not checked yetcites [167] Chen, C., et al. Progressive Feature Alignment for Unsupervised Domain Adaptation. arXiv:1811.08585 (2019) source
- Not checked yetcites [168] Jin, X., et al.: Feature Alignment and Restoration for Domain Generalization and Adaptation. arXiv:2006.12009 (2020)
- Not checked yetcites [165] Ščupáková, K., et al.: A patch-based super resolution algorithm for improving image resolution in clinical mass spectrometry. Sci. Rep. 9, 2915 (2019) source
- Not checked yetcites [166] Bashiri, F.S., et al.: Multi-modal medical image registration with full or partial data: a manifold learning approach. J. Imag. 5, 5 (2019) source
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The main properties that distinguish CNNs from other models include their ability to capture local connectivity between units, to share weights across layers, and to block a sequence of hidden layers [4].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [4] LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“To this end, several popular variants have been developed to efficiently handle this constraint and the gradient vanishing problem with impressive results, including long short-term memory (LSTM) [13] and gated recurrent linear units (GRU) [14].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [13] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997) source
- Not checked yetcites [14] Li, S., Li, W., Cook, C., Zhu, C., Gao, Y.: Independently recurrent neural network (IndRNN): building a longer and deeper RNN. arXiv:1803.04831 (2018) source
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Recently, with the advent of robust and low-cost RGB-D sensors such as the Kinect, the computer vision community has turned its attention to integrating RGB images and corresponding depth maps (2.5D) into multimodal architectures as shown in Fig. 7.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Convolutional neural networks (CNNs or ConvNets) are a class of deep feed-forward neural networks whose main purpose is to extract spatial patterns from visual input signals [20, 22].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [20] Khan, A., Sohail, A., Zahoora, U., Qureshi, A.S.: A survey of the recent architectures of deep convolutional neural networks. Artif Intell Rev (2020) source
- Not checked yetcites [22] Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2015) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, a multimodal CNN serves as a powerful feature extractor that learns local cross-modal features from visual modalities.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In contrast to single-task learning, the idea behind this paradigm is to learn a shared representation that can be used to respond to several tasks in order to ensure better generalizability.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multitask learning
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A formalism for integrating an AM into the basic RNN model was developed by Bahdanau et al. [1].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [1] Bahdanau, D., Cho, K., Bengio, Y.: Neural Machine Translation by Jointly Learning to Align and Translate. arXiv:1409.0473 (2016)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Soft parameter sharing [111]: It consists of extracting a set of feature vectors and simultaneously drawing similarity relationships between them.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multitask learning
What the sources say
- Not checked yetcites [111] Duong, L., Cohn, T., Bird, S., Cook, P. low resource dependency parsing: cross-lingual parameter sharing in a neural network parser. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 845–850 (2015) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Since the encoding side of an RNN generates a fixed-length feature vector from its input sequence, this can lead to very tedious and time-consuming parameter tuning.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Historically, they were first developed by Goodfellow et al. [16], which demonstrated the ability to generate realistic and reasonably impractical representations from noisy data domains.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Fusion algorithms
What the sources say
- Not checked yetcites [16] Goodfellow, I.J., et al.: Generative adversarial networks. arXiv:1406.2661 (2014)
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The task has attracted a lot of interest because of its enormous relevance in many real-world applications, including video surveillance [82], autonomous driving [83], etc.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [82] 1.Mohanapriya, D., Mahesh, K.: Chapter 5—an efficient framework for object tracking in video surveillance. In: The Cognitive Approach in Cloud Computing and Internet of Things Technologies for Surveillance Tracking Systems, pp. 65–74 (2020) source
- Not checked yetcites [83] Rangesh, A., Trivedi, M.M.: No blind spots: full-surround multi-object tracking for autonomous vehicles using cameras and LiDARs. IEEE Trans. Intelli. Veh. 4, 588–599 (2019) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Transfer learning (TL) [70] is one of the model regularization techniques that have proven their effectiveness for training deep models with a limited amount of available data and avoiding overfitting problems.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal transfer learning
What the sources say
- Not checked yetcites [70] Shao, L., Zhu, F., Li, X.: Transfer learning for visual categorization: a survey. IEEE Trans. Neural Netw. Learn. Syst. 26, 1019–1034 (2015) source
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Training a deep model from scratch requires a large amount of labeled data to achieve an acceptable level of performance.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“High-precision object recognition systems with multiple sensors are aware of external noise and environmental sensitivity (e.g., lighting variations, occlusion, etc.).”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“This function allows training a one-stage detector with high accuracy by reducing the level of artifacts.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is a cascaded version of R-CNN aimed at achieving a better compromise between object localization and classification.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The objective is to project a set of synthesized features in order to make the model more generalizable toward the recognition of the unseen class in test samples [135].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Zero-shot learning
What the sources say
- Not checked yetcites [135] Liu, Y., Tuytelaars, T.: A: deep multi-modal explanation model for zero-shot learning. IEEE Trans. Image Process. 29, 4788–4803 (2020) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The amount of labeled data samples for effective model training is often insufficient to recognize all possible object categories in an image.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In order to significantly reduce this complexity, the deep learning community has pioneered a new generation of CNN-based frameworks.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, due to their complexity, this type of solution is not cost-effective, both in terms of training duration and memory consumption.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, MOT methods are mainly designed to optimize the dynamic matching of the objects of interest detected in each frame.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As mentioned before, two-stage detectors are generally based on a combination of a CNN model to perform classification and a patch proposal module to generate candidate regions like RPNs.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Multi-object tracking (MOT) has been extensively explored in recent literature for a wide range of applications [86, 138].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [86] Ciaparrone, G., et al.: Deep learning in video multi-object tracking: a survey. Neurocomputing 381, 61–88 (2020) source
- Not checked yetcites [138] Zhang, X., et al.: Online multi-object tracking with pedestrian re-identification and occlusion processing. Vis. Comput. (2020). https://doi.org/10.1007/s00371-020-01854-0 source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The attentional alignment task is based on the attentional mapping between the features of the input modality and the target one, while the semantic alignment task takes the form of an alignment method that directly provides alignment capabilities to a predictive model.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal alignment
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In contrast, SSD (single-shot multibox detector) [78] has reduced the needs of the patches’ proposal network and, thus, accelerated the object detection pipeline.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [78] Liu, W. et al.: SSD: single shot multibox detector, pp. 21–37. arXiv:1512.02325 (2016) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In other words, it addresses multi-class learning problems when some classes do not have sufficient training data.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Zero-shot learning
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The amount of labeled data samples for effective model training is often insufficient to recognize all possible object categories in an image.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When modeling multimodal data, several compromises have to be made between system performance, computational burden, and processing speed.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Multimodal MOT is a universal aspect of MOT aimed at ensuring the accuracy of autonomous systems by mapping the motion sequence of dynamic objects [194].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [194] Zhang, W., et al.: Robust Multi-Modality Multi-Object Tracking. arXiv:1909.03850 (2019) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the context of multimodality, a multimodal mapping scheme typically combines visual and semantic attributes using only data related to the seen classes.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Zero-shot learning
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The amount of labeled data samples for effective model training is often insufficient to recognize all possible object categories in an image.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These techniques have proven effective for the accurate detection of multiple objects under normal and extreme environmental conditions.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“The recent literature has shown that it is attractive to learn shared representations from the complementarity and synergies between several modalities for increasing the discriminatory power of models [190].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [190] Michael, Y.B., Rosenhahn, V.M.: Multimodal Scene Understanding, 1st edn. Academic Press, London (2019)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this respect, two-stage detectors are generally slow in terms of real-time processing.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: In some vision applications, such as autonomous driving, security, and video surveillance, real-time conditions become necessary.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In practice, there are several possible detection methods. The most typical of these is to apply the classifier to an arbitrary region of the image or to a range of different shapes and scales.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“As similar to the principle of YOLO and SSD detectors, RetinaNet [79] takes only one stage to detect dense objects by producing multi-scale semantic feature maps using a feature pyramid network (FPN) backbone and the ResNet model.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [79] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. arXiv:1708.02002 (2018) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, the learning process can learn the data distribution in a non-parallel manner and ensure its synchronization over time.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal transfer learning
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Rests on an assumption: Training a deep model from scratch requires a large amount of labeled data to achieve an acceptable level of performance.
None of the sources cited for this claim have been checked yet.
In the paper
“This is why zero-shot learning [130] takes place. This supervised learning approach opens up many valuable applications such as object detection [131], object classification and retrieval of videos [141], especially when appropriate datasets are missing.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Zero-shot learning
What the sources say
- Not checked yetcites [130] Wang, W., et al.: A survey of zero-shot learning: settings, methods, and applications. ACM Trans. Intell. Syst. Technol. 10, 13:1–13:37 (2019)
- Not checked yetcites [131] Wei, L., et al.: A single-shot multi-level feature reused neural network for object detection. Vis. Comput. (2020). https://doi.org/10.1007/s00371-019-01787-3 source
- Not checked yetcites [141] Parida, K., et al.: Coordinated joint multimodal embeddings for generalized audio-visual zero-shot classification and retrieval of videos. In: CVPR, pp. 3251–3260 (2020) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Rests on an assumption: The amount of labeled data samples for effective model training is often insufficient to recognize all possible object categories in an image.
In the paper
“Cascade R-CNN [26] is one of the most effective multi-stage detectors that have proven their robustness over one and two-stage methods.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [26] Cai, Z., Vasconcelos, N.: Cascade R-CNN: high quality object detection and instance segmentation. arXiv:1906.09756 (2019) source
Strength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In general, multimodality has been employed in many vision tasks and applications, such as face recognition and image retrieval.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“More recently, the availability of low-cost and robust sensors (e.g., RGB-D sensors, stereo, etc.) has encouraged the computer vision community to focus on combining the RGB modality with other sensing modalities.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, the most common approach is to concatenate heterogeneous features from different modalities to generate an artificial multimodal representation.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Such methods tend to use GAN models to synthesize and reconstruct the visual features of the unseen classes, resulting in high accuracy classification and ensuring a balance between seen and unseen class labels [136, 137].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Zero-shot learning
What the sources say
- Not checked yetcites [136] Xian, Y., Lorenz, T., Schiele, B., Akata, Z.: Feature generating networks for zero-shot learning. In: CVPR, pp. 5542–5551 (2018) source
- Not checked yetcites [137] Kumar, Y. et al.: Harnessing GANs for Zero-shot Learning of New Classes in Visual Speech Recognition. arXiv:1901.10139 (2020) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The amount of labeled data samples for effective model training is often insufficient to recognize all possible object categories in an image.
In the paper
“To deal with the class imbalance in the training phase, a novel loss function called “focal loss” is considered by [79].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [79] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. arXiv:1708.02002 (2018) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most popular use of semantic alignment is to create a dataset with associated labels and then generate a semantically aligned dataset.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal alignment
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both of these tasks have proven effective in multimodal alignment, where attentional alignment features are better able to take into account the long-term dependencies between different concepts.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal alignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, multi-object detection in both indoor and outdoor environments under varying environmental and lighting conditions remains one of the major challenges facing the computer vision community.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It can learn multi-scale feature representation from multi-resolution images.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Over the last few decades, most deep learning-based object tracking systems have been based on CNN architectures [84, 139].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [84] Liu, L., et al.: Deep learning for generic object detection: a survey. Int. J. Comput. Vis. 128, 261–318 (2020) source
- Not checked yetcites [139] Abbass, M.Y., et al.: Efficient object tracking using hierarchical convolutional features model and correlation filters. Vis. Comput. (2020). https://doi.org/10.1007/s00371-020-01833-5 source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For decades, visual tracking has been one of the major challenges for the computer vision community.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its capability to detect objects at different scales enables it to enhance the robustness of the entire chain.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The attentional alignment task is based on the attentional mapping between the features of the input modality and the target one, while the semantic alignment task takes the form of an alignment method that directly provides alignment capabilities to a predictive model.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Deep multimodal learning architectures > Multimodal alignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In [160], Wang et al. introduced a residual CNN architecture and loss network to transfer the artistic style of the input picture across multiple scales and dimensions.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [160] Wang, X., et al.: Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer. arXiv:1612.01895 (2017) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Similarly, Peri et al. [53] proposed a multimodal framework that encodes both images and captions using CNN and RNN as an intermediate level representation and then decodes these multimodal representations into a new caption that is similar to the input.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [53] Peri, D., Sah, S., Ptucha, R.: Show, Translate and Tell. arXiv:1903.06275 (2019) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The challenge of action recognition is mainly related to the difficulty of extracting body silhouettes from foreground rigid objects to focus on their emotions [96].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [96] Escalera, S., Athitsos, V., Guyon, I.: Challenges in multimodal gesture recognition. J. Mach. Learn. Res. 17, 1–54 (2016)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“For instance, RGB-D cameras (e.g., Microsoft Kinect, Asus Xtion, and so on) can provide color and pixel-wise depth information, characterizing the distance of visual objects in a complex scene [199].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [199] Shamwell, E.J., et al.: Unsupervised deep visual-inertial odometry with online error correction for RGB-D imagery. IEEE Trans. Pattern Anal. Mach. Intell. (2019). https://doi.org/10.1109/TPAMI.2019.2909895 source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The authors of [159] proposed a universal graph-based style transfer to transform multimodal features by matching style patterns and semantic content and appearance in a way that avoids the lack of flexibility in real-world scenarios.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [159] Zhang, Y. et al.: Multimodal style transfer via graph cuts. In: ICCV, pp. 5943–5951 (2019) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Note that face detection, face identification, and face reconstruction are subtasks of face recognition [50].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [50] Wang, M., Deng, W.: Deep Face Recognition: A Survey. arXiv:1804.06655 (2019)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The authors of [128] presented an unsupervised image captioning framework based on a new alignment method that allows the simultaneous integration of visual and textual streams through semantic learning of multimodal embeddings of the language and vision domains.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [128] Laina, I., et al.: Towards unsupervised image captioning with shared multimodal embeddings. In: ICCV, pp. 7414–7424 (2019) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The authors of [93] introduced a biometric system that combines biometric traits from different modalities (face and iris) to establish an individual’s identity.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [93] Ammour, B., Boubchir, L., Bouden, T., Ramdani, M.: Face-Iris multimodal biometric identification system. Electronics 9, 85 (2020) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“It is essentially a weighted sum of the error (loss) between the input content and the output image and the loss between the original style and the applied style [161].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [161] Jing, Y., et al.: Neural Style Transfer: A Review. arXiv:1705.04058 (2018)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Most recently, Biten et al. [52] combined both visual and textual data to generate captions across two stages: template caption generation stage and entity insertion stage.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [52] Biten, A.F., Gomez, L., Rusiñol, M., Karatzas, D.: Good News, Everyone! Context driven entity-aware captioning for news images. arXiv:1904.01475 (2019) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For instance, the Mask R-CNN model offers the possibility of locating instances of objects with class labels and segmenting them with semantic masks.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In [73], Koo et al. developed a multimodal human recognition framework based on face and body information extracted from deep CNNs. They employed the late fusion policy to merge the high-level features across the different modalities.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [73] Koo, J.H., Cho, S.W., Baek, N.R., Kim, M.C., Park, K.R.: CNN-based multimodal human recognition in surveillance environments. Sensors 18, 3040 (2018) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Regarding the multimodal dimensions of gesture recognition, the authors of [97] proposed a multi-stream architecture based on the RNN (LSTM) model to capture spatial-temporal features from gesture data.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [97] Nishida, N., Nakayama, H.: Multimodal gesture recognition using multi-stream recurrent neural network. In: Revised Selected Papers of the 7th Pacific-Rim Symposium on Image and Video Technology, pp. 682–694 (2015) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In recent years, deep learning algorithms have been developed to save time and dependability during patient care by improving clinical accuracy and detecting abnormalities in medical images [104, 140].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [104] Xu, Y.: Deep learning in multimodal medical image analysis. In: Health Information Science, pp. 193–200 (2019) source
- Not checked yetcites [140] Xi, P.: An integrated approach for medical abnormality detection using deep patch convolutional neural networks. Vis. Comput. 36, 1869–1882 (2020) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Gatys et al. [156] demonstrated that visual features of models could be combined to represent image styles.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [156] Gatys, L., et al.: A Neural Algorithm of Artistic Style. arXiv:1508.06576 (2015) source
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Therefore, each modality will be covered individually using adaptive approaches. After joining the multi-domain features, the high-level classification will be performed by an individual classifier.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“In [48], the structure of deep multimodal Boltzmann machines (DMBM) [71] is similar to that of DBM, but it can admit more than one modality.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [48] Alam, M. R., Bennamoun, M., Togneri, R., Sohel, F.: A deep neural network for audio-visual person recognition. In: IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–6 (2015) source
- Not checked yetcites [71] Srivastava, N., Salakhutdinov, R.: Multimodal learning with deep Boltzmann machines. J. Mach. Learn. Res. 15(1), 2949–2980 (2014)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Over the last few years, the fully convolutional network (FCN) [29] has become one of the robust models for a wide range of image types (multimedia, aerial, medical, etc.).”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Generic computer vision tasks
What the sources say
- Not checked yetcites [29] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. arXiv:1411.4038 (2015) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“According to Salakhutdinov et al. [72], a DBM is a generative model that includes several layers of hidden variables.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [72] Salakhutdinov, R., Hinton, G.: Deep Boltzmann machines. In: Artificial Intelligence and Statistics, pp. 448–455 (2009)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In [142], Lee et al. proposed a new CNN-based retinal image registration method to learn multimodal features simultaneously from several imaging modalities.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [142] Lee, J. A., et al.: Deep step pattern representation for multimodal retinal image registration. In: CVPR, pp. 5077–5086 (2019) source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In [98], the authors developed a multimodal gesture recognition system using the 3D Residual CNN (ResC3D) model [99] trained on an RGB-D dataset.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [98] Miao, Q. et al.: Multimodal gesture recognition based on the ResC3D network. In: IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 3047–3055 (2017) source
- Not checked yetcites [99] Tran, D., Ray, J., Shou, Z., Chang, S.-F., Paluri, M.: ConvNet architecture search for spatiotemporal feature learning. arXiv:1708.05038 (2017)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In general, capturing multimodal data streams under different acquisition conditions and increasing the data volume makes it easier to recognize visual content.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, it is important to note that each multimodal problem requires a specific fusion strategy, ranging from traditional methods to deep learning techniques.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Conclusion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the need to reduce computing capacity remains the main challenge, which can lead to a deterioration in the overall accuracy of training algorithms.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, networks trained with a limited number of examples can no longer generalize well to a new dataset.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“For instance, the edge/cloud computing solution for mulitmodal analysis provides an effortless way to create and handle multimodal datasets for training and deploying models [107].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
What the sources say
- Not checked yetcites [107] Wang, X., et al.: Convergence of edge computing and deep learning: a comprehensive survey. IEEE Commun. Surv. Tutorials 1, 2020 (2020)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Deep learning models are often robust strategies for dealing with the linear and nonlinear combination of multimodal data.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The amount of multimodal data significantly affects the overall performance of the vision system.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, feature-level fusion is more flexible than decision-level fusion due to the homogeneity of data samples.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The deep multimodal analysis relies on a large amount of heterogeneous sensor data to achieve high performance and avoid overfitting problems.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Popular visual multimodal datasets
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In practice, autonomous vehicles, healthcare robots, and other real-time embedded systems consume more hardware resources, storage, and battery than other emerging technologies, resulting in a lack of adaptation to future needs.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In other words, acquiring and processing a large amount of multimodal data is costly in terms of computation complexity and memory consumption.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, choosing the right fusion of different schemes remains a vital challenge for the computer vision community in terms of accuracy and efficiency.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Conclusion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Also, AE can address missing patterns by generating intermediate shared representations from the input data and showing intra- and inter-pattern correlations.”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Discussion, limitations, and challenges
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Among the advantages of these types of sensors are their low computational cost, their long-range, their ability to have an internal mechanism to limit the impact of bad weather, etc. [149].”
§ A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets > Tasks and applications > Multimodal applications
What the sources say
- Not checked yetcites [149] Van Brummelen, J., et al.: Autonomous vehicle perception: the technology of today and tomorrow. Transp. Res. C Emerg. Technol. 89, 384–406 (2018) source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper relies on this without support.
A majority of contemporary machine learning applications use multiple modalities, such as visual and textual data.
“However, most modern machine learning applications involve more than one modality (e.g., visual and textual modalities), such as embodied question answering, vision-and-language navigation, etc.”
The paper relies on this without support.
The absence of labeled multimodal data in the literature can reduce flexibility and accuracy.
“Moreover, the lack of labeled multimodal data in the current literature can lead to reduced flexibility and accuracy, often requiring cooperation between different modalities.”
The paper relies on this without support.
Using multiple sensors as multimodal data sources yields complementary and richer information and can outperform a single input.
“The main reason for using multimodal data sources is that it is possible to extract complementary and richer information coming from multiple sensors, which can provide much more optimistic results than a single input.”
The paper relies on this without support.
Deep multimodal learning's success has been driven by large multimodal datasets, powerful GPU-equipped computers, and high-quality multi-scale feature representations.
“The success of deep multimodal learning techniques has been driven by many factors that have led many researchers to adopt these methods to improve model performance. These factors include large volumes of widely usable multimodal datasets, more powerful computers with fast GPUs, and high-quality feature representation at multiple scales.”
The paper relies on this without support.
Training a deep model from scratch requires a large amount of labeled data to achieve an acceptable level of performance.
“Typically, training a deep model from scratch requires a large amount of labeled data to achieve an acceptable level of performance.”
The paper relies on this without support.
The amount of labeled data samples for effective model training is often insufficient to recognize all possible object categories in an image.
“In practice, the amount of labeled data samples for effective model training is often insufficient to recognize all possible object categories in an image (i.e., seen and unseen classes).”
The paper relies on this without support.
In some vision applications, such as autonomous driving, security, and video surveillance, real-time conditions become necessary.
“However, in some vision applications, such as autonomous driving, security, video surveillance, etc., real-time conditions become necessary.”
The paper relies on this without support.
Most facial recognition systems, including biometric systems, are limited by a lack of universality and by variations in the appearance of visual patterns.
“However, most facial recognition systems, including biometric systems, suffer from a restriction in terms of universality and variations in the appearance of visual patterns.”
The paper relies on this without support.
Most current scene-content analysis methods cannot handle information that is not usable in real-life contexts.
“However, most current scene-content analysis methods are still limited in their ability to deal with information that is not usable in real-life contexts.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Developments in router Zloss regularization techniques plays a crucial role in addressing expert imbalance in MoE models by fine-tuning the gating mechanism, ensuring a more equitable workload distribution across experts and fostering a stable training environment, thereby enhancing model performance and reducing training time and computational overhead [301], [302].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [301]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Application-table continuation is labeled Table 3
MinorThe rows under "Table 3 continued" list applications, fusion schemes, and results, continuing the material identified as Table 2 rather than the dataset table described as Table 3.
“Table: Table 3 continued | [142] | 2019 | Medical data analysis | RGB + FA + OCT | Alignment | Color fundus-FA (ACC = 90.10%) |”
“Table: Table 2 Summary of the multimodal applications reviewed, their related technical details, and best results achieved”
CNN introduction dated both 1998 and 2012
MinorThe paper describes LeCun et al.'s convolutional neural networks in 1998, but later says the CNN model was introduced by Krizhevsky et al. in 2012.
“Since its introduction by Krizhevsky et al. [7] in 2012, the CNN model has been successfully applied”
“such as the convolutional neural networks of LeCun et al. [21] in 1998 and the spectacular deep network results of Krizhevsky et al. [7] in 2012”
Citations
Cites
Items 201–280 of 280
Multi-modal Unsupervised Feature Learning for RGB-D Scene Labeling
Wang, A., Lu, J., Wang, G., Cai, J., & Cham, T.-J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_30 · Cited by 50
Bayoudh, K., Hamdaoui, F., & Mtibaa, A. (2020) · Physical and Engineering Sciences in Medicine · doi:10.1007/s13246-020-00957-1 · Cited by 50
Self-citationParida, K. K., Matiyali, N., Guha, T., & Sharma, G. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093438 · Cited by 49
Attention Strategies for Multi-Source Sequence-to-Sequence Learning
Libovický, J., & Helcl, J. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2031 · Cited by 48
Xi, P., Guan, H., Shu, C., Borgeat, L., & Goubran, R. (2020) · The Visual Computer · doi:10.1007/s00371-019-01775-7 · Cited by 47
Content-Based Image Retrieval Research
Duan, G., Yang, J., & Yang, Y. (2011) · Physics Procedia · doi:10.1016/j.phpro.2011.11.073 · Cited by 46
Multi-Modal Medical Image Registration with Full or Partial Data: A Manifold Learning Approach
Bashiri, F. S., Baghaie, A., Rostami, R., Yu, Z., & D’Souza, R. M. (2018) · Journal of Imaging · doi:10.3390/jimaging5010005 · Cited by 46
Video Question Answering with Spatio-Temporal Reasoning
Jang, Y., Song, Y., Kim, C. D., Yu, Y., Kim, Y., & Kim, G. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01189-x · Cited by 46
Learning Target-Oriented Dual Attention for Robust RGB-T Tracking
Yang, R., Zhu, Y., Wang, X., Li, C., & Tang, J. (2019) · 2019 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2019.8803528 · Cited by 45
Fan, X., Jiang, W., Luo, H., & Mao, W. (2022) · The Visual Computer · doi:10.1007/s00371-020-02015-z · Cited by 45
Multimodal attention networks for low-level vision-and-language navigation
Landi, F., Baraldi, L., Cornia, M., Corsini, M., & Cucchiara, R. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103255 · Cited by 44
A Layer-Based Sequential Framework for Scene Generation with GANs
Turkoglu, M. O., Thong, W., Spreeuwers, L., & Kicanaoglu, B. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018901 · Cited by 41
SVM-Based Spectrum Sensing in Cognitive Radio
Zhang, D., & Zhai, X. (2011) · International Conference on Wireless Communications, Networking and Mobile Computing · doi:10.1109/wicom.2011.6040028 · Cited by 38
Wang, X. E., Huang, Q., Çelikyılmaz, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.10092 · Cited by 37
Multi-modal gated recurrent units for image description
Li, X., Yuan, A., & Lu, X. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5856-1 · Cited by 36
DTR-HAR: deep temporal residual representation for human activity recognition
Basly, H., Ouarda, W., Sayadi, F. E., Ouni, B., & Alimi, A. M. (2022) · The Visual Computer · doi:10.1007/s00371-021-02064-y · Cited by 36
Online multi-object tracking with pedestrian re-identification and occlusion processing
Zhang, X., Wang, X., & Gu, C. (2021) · The Visual Computer · doi:10.1007/s00371-020-01854-0 · Cited by 35
Perceive, Transform, and Act: Multi-Modal Attention Networks for Vision-and-Language NavigationPreprint
Landi, F., Baraldi, L., Cornia, M., Corsini, M., & Cucchiara, R. (2019) · arXiv.org · doi:10.48550/arxiv.1911.12377 · Cited by 35
Multimodal Recurrent Neural Networks With Information Transfer Layers for Indoor Scene Labeling
Abdulnabi, A. H., Shuai, B., Zuo, Z., Chau, L.-P., & Wang, G. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2774007 · Cited by 33
Real-Time Driving Scene Semantic Segmentation
Wang, W., Fu, Y., Pan, Z., Li, X., & Zhuang, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2975640 · Cited by 33
Deep Learning in Multimodal Medical Image Analysis
Xu, Y. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32962-4_18 · Cited by 32
Complete 3D Scene Parsing from an RGBD Image
Zou, C., Guo, R., Li, Z., & Hoiem, D. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1133-z · Cited by 31
A Deep Multi-Modal Explanation Model for Zero-Shot Learning
Liu, Y., & Tuytelaars, T. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2975980 · Cited by 31
CNN-Based Multimodal Human Recognition in Surveillance Environments
Koo, J. H., Cho, S. W., Baek, N. R., Kim, M. C., & Park, K. R. (2018) · Sensors · doi:10.3390/s18093040 · Cited by 30
A single-shot multi-level feature reused neural network for object detection
Wei, L., Cui, W., Hu, Z., Sun, H., & Hou, S. (2021) · The Visual Computer · doi:10.1007/s00371-019-01787-3 · Cited by 29
Optical Flow Estimation with Deep Learning, a Survey on Recent Advances
Savian, S., Elahi, M., & Tillo, T. (2020) · Unsupervised and Semi-Supervised Learning · doi:10.1007/978-3-030-32583-1_12 · Cited by 29
Multiple answers to a question: a new approach for visual question answering
Hosseinabad, S. H., Safayani, M., & Mirzaei, A. (2020) · The Visual Computer · doi:10.1007/s00371-019-01786-4 · Cited by 28
Perceive, Transform, and Act: Multi-Modal Attention Networks for Vision-and-Language NavigationPreprint
Landi, F., Baraldi, L., Cornia, M., Corsini, M., & Cucchiara, R. (2019) · arXiv (Cornell University) · Cited by 27
Güçlü, O., & Can, A. B. (2019) · The Visual Computer · doi:10.1007/s00371-019-01720-8 · Cited by 24
Memory influences haptic perception of softness
Metzger, A., & Drewing, K. (2019) · Scientific Reports · doi:10.1038/s41598-019-50835-4 · Cited by 24
Efficient object tracking using hierarchical convolutional features model and correlation filters
Abbass, M. Y., Kwon, K.-C., Kim, N., Abdelwahab, S. A., El-Samie, F. E. A., & Khalaf, A. A. M. (2021) · The Visual Computer · doi:10.1007/s00371-020-01833-5 · Cited by 24
Attentional Alignment Networks.
Yue, L., Miao, X., Wang, P., Zhang, B., Zhen, X., & Cao, X. (2018) · British Machine Vision Conference (BMVC) · Cited by 23
Ščupáková, K., Terzopoulos, V., Jain, S., Smeets, D., & Heeren, R. M. A. (2019) · Scientific Reports · doi:10.1038/s41598-019-38914-y · Cited by 23
Accurate and Robust Monocular SLAM with Omnidirectional Cameras
Liu, S., Guo, P. F., Feng, L., & Yang, A. (2019) · Sensors · doi:10.3390/s19204494 · Cited by 23
SAR2OPT: Image Alignment Between Multi-Modal Images Using Generative Adversarial Networks
Toriya, H., Dewan, A. M., & Kitahara, I. (2019) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2019.8898605 · Cited by 23
Real-time multimodal ADL recognition using convolution neural networks
Madhuranga, D., Madushan, R., Siriwardane, C., & Gunasekera, K. (2020) · The Visual Computer · doi:10.1007/s00371-020-01864-y · Cited by 23
Paradigm shifts in super-resolution techniques for remote sensing applications
Rohith, G., & Kumar, L. S. (2020) · The Visual Computer · doi:10.1007/s00371-020-01957-8 · Cited by 23
$$\hbox {S}^2\hbox {RGAN}$$: sonar-image super-resolution based on generative adversarial network
Song, H., Wang, M., Zhang, L., Li, Y., Jiang, Z., & Yin, G. (2021) · The Visual Computer · doi:10.1007/s00371-020-01986-3 · Cited by 23
Progressive Feature Alignment for Unsupervised Domain AdaptationPreprint
Chen, C., Xie, W., Huang, W., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08585 · Cited by 23
Attention-Based Multimodal Fusion for Video DescriptionPreprint
Hori, C., Hori, T., Lee, T., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.03126 · Cited by 22
Controllable Video Captioning with POS Sequence Guidance Based on Gated Fusion NetworkPreprint
Wang, B., Ma, L., Zhang, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.10072 · Cited by 22
Combining CNN streams of dynamic image and depth data for action recognition
Singh, R. K., Khurana, R., Kushwaha, A. K. S., et al. (2020) · Multimedia Systems · doi:10.1007/s00530-019-00645-5 · Cited by 21
Multimodal Scene Understanding
(2019) · Elsevier eBooks · doi:10.1016/c2018-0-01791-0 · Cited by 20
Zarbakhsh, P., & Demirel, H. (2019) · The Visual Computer · doi:10.1007/s00371-019-01705-7 · Cited by 17
VideoNavQA: Bridging the Gap between Visual and Embodied Question AnsweringPreprint
Cangea, C., Belilovsky, E., Lió, P., & Courville, A. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04950 · Cited by 17
Deep Multimodal Feature Analysis for Action Recognition in RGB+D VideosPreprint
Shahroudy, A., Ng, T.-T., Gong, Y., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.07120 · Cited by 16
Multi-level progressive parallel attention guided salient object detection for RGB-D images
Liu, Z., Duan, Q., Shi, S., & Zhao, P. (2021) · The Visual Computer · doi:10.1007/s00371-020-01821-9 · Cited by 16
Ma, T., & Tian, W. (2021) · The Visual Computer · doi:10.1007/s00371-020-01843-3 · Cited by 16
Wang, L., Wang, Z., Yang, X., Hu, S., & Zhang, J. (2018) · The Visual Computer · doi:10.1007/s00371-018-1609-4 · Cited by 15
Kandylakis, Z., Vasili, K., & Karantzalos, K. (2019) · Remote Sensing · doi:10.3390/rs11040446 · Cited by 15
Latent Multi-Task Architecture Learning
Ruder, S., Bingel, J., Augenstein, I., & Søgaard, A. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014822 · Cited by 14
Feature Learning Using Stacked Autoencoder for Shared and Multimodal Fusion of Medical Images
Singh, V., Verma, N. K., Ul Islam, Z., & Cui, Y. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-981-13-1132-1_5 · Cited by 13
TICS: text–image-based semantic CAPTCHA synthesis via multi-condition adversarial learning
Jia, X., Xiao, J., & Wu, C. (2022) · The Visual Computer · doi:10.1007/s00371-021-02061-1 · Cited by 13
RGB-D-based gaze point estimation via multi-column CNNs and facial landmarks global optimization
Zhang, Z., Lian, D., & Gao, S. (2020) · The Visual Computer · doi:10.1007/s00371-020-01934-1 · Cited by 12
Principal Component Analysis and Factor Analysis
(2004) · The MIT Press eBooks · doi:10.7551/mitpress/3717.003.0017 · Cited by 12
Robust Multi-Modality Multi-Object TrackingPreprint
Zhang, W., Zhou, H., Sun, S., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03850 · Cited by 12
Fine-grained talking face generation with video reinterpretation
Huang, X., Wang, M., & Gong, M. (2021) · The Visual Computer · doi:10.1007/s00371-020-01982-7 · Cited by 11
Wang, X., Oxholm, G., Zhang, D., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.01895 · Cited by 10
Online learning for automatic segmentation of 3D data
Tombari, F., Stefano, L. D., & Giardino, S. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6094649 · Cited by 9
Cutting Edge: Soft Correspondences in Multimodal Scene Parsing
Namin, S. T., Najafi, M., Salzmann, M., & Petersson, L. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.141 · Cited by 9
Semantic embeddings of generic objects for zero-shot learning
Hascoet, T., Ariki, Y., & Takiguchi, T. (2019) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-018-0371-x · Cited by 9
Adversarial Image Registration with Application for MR and TRUS Image FusionPreprint
Yan, P., Xu, S., Rastinehad, A. R., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.11024 · Cited by 9
A deep neural network for audio-visual person recognition
Alam, M. R., Bennamoun, M., Togneri, R., & Sohel, F. (2015) · 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas.2015.7358754 · Cited by 8
Multimodal deep learning approach for joint EEG-EMG data compression and classificationPreprint
Said, A. B., Mohamed, A. M. S., Elfouly, T., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08970 · Cited by 8
Dapogny, A., Bailly, K., & Cord, M. (2020) · 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) · doi:10.1109/fg47880.2020.00038 · Cited by 8
Video Object Detection Base on RGB and Optical Flow Analysis
Zhang, S., Wang, T., Wang, C., Wang, Y., Shan, G., & Snoussi, H. (2019) · 2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI) · doi:10.1109/cchi.2019.8901921 · Cited by 7
An Efficient Framework for Object Tracking in Video Surveillance
Mohanapriya, D., & Mahesh, K. (2020) · The Cognitive Approach in Cloud Computing and Internet of Things Technologies for Surveillance Tracking Systems · doi:10.1016/b978-0-12-816385-6.00005-2 · Cited by 7
Transfer learning for multimodal dialog
Palaskar, S., Sanabria, R., & Metze, F. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101093 · Cited by 6
Quality and Relevance Metrics for Selection of Multimodal Pretraining Data
Rao, R., Rao, S., Nouri, E., Dey, D., Celikyilmaz, A., & Dolan, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00486 · Cited by 6
Transformation of portraits to Picasso’s cubism style
Lian, G., & Kang, Z. (2019) · The Visual Computer · doi:10.1007/s00371-019-01661-2 · Cited by 4
Peri, D., Sah, S., & Ptucha, R. (2019) · 2019 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2019.8802922 · Cited by 4
Bidirectional Multimodal Recurrent Neural Networks with Refined Visual Features for Image Captioning
Shu, Y., Zhang, L., Li, Z., & Tang, J. (2018) · Communications in computer and information science · doi:10.1007/978-981-10-8530-7_8 · Cited by 3
Semantic Alignment: Finding Semantically Consistent Ground-truth for Facial Landmark DetectionPreprint
Liu, Z., Zhu, X., Hu, G., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10661 · Cited by 3
Diagnosing the Environment Bias in Vision-and-Language NavigationPreprint
Zhang, Y., Tan, H. X., & Bansal, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.03086 · Cited by 3
Multimodal Deep Domain AdaptationPreprint
Bucci, S., Loghmani, M. R., & Barbara Caputo, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.11697 · Cited by 2
Harnessing GANs for Zero-Shot Learning of New Classes in Visual Speech Recognition
Kumar, Y., Sahrawat, D., Maheshwari, S., Mahata, D., Stent, A., Yin, Y., Ratn Shah, R., & Zimmermann, R. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5649 · Cited by 1
A Layer-Based Sequential Framework for Scene Generation with GANsPreprint
Türkoglu, M. Ö., Thong, W. E., Spreeuwers, L., & Kicanaoglu, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00671 · Cited by 1
Show, Translate and TellPreprint
Peri, D., Sah, S., & Ptucha, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.06275
Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition with Multimodal TrainingPreprint
Abavisani, M., Joze, H. R. V., & Patel, V. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.06145
Flagged first, then by citations
Research works printed but not linked: 6
- Bayoudh, K.: From machine learning to deep learning, (1st ed.), Ebook, ISBN: 9781387465606 (2017)
- Wang, M., Deng, W.: Deep Face Recognition: A Survey. arXiv:1804.06655 (2019) · arXiv
- Salakhutdinov, R., Hinton, G.: Deep Boltzmann machines. In: Artificial Intelligence and Statistics, pp. 448–455 (2009)
- Escalera, S., Athitsos, V., Guyon, I.: Challenges in multimodal gesture recognition. J. Mach. Learn. Res. 17, 1–54 (2016)
- Yue, L., et al.: Attentional alignment network. In: BMVC (2018)
- Srivastava, N., et al.: Multimodal learning with deep Boltzmann machines. J. Mach. Learn. Res. 15, 2949–2980 (2014)
Cited by
Sorted by citations
Effective Techniques for Multimodal Data Fusion: A Comparative Analysis
Pawłowski, M., Wróblewska, A., & Sysko-Romańczuk, S. (2023) · Sensors · doi:10.3390/s23052381 · Cited by 180
McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102
“…the field of multimodal learning still confronts ongoing challenges, such as refining the architectures to handle diverse data types more effectively [269], [270], developing comprehensive datasets that accurately represent multifaceted information [269], [271], and establishing benchmarks for…”
BackgroundA systematic review of intermediate fusion in multimodal deep learning for biomedical applications
Guarrasi, V., Aksu, F., Caruso, C. M., Di Feola, F., Rofena, A., Ruffini, F., & Soda, P. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105509 · Cited by 95
Detection in adverse weather conditions for autonomous vehicles via deep learning
Al-Haija, Q. A., Gharaibeh, M., & Odeh, A. (2022) · AI · doi:10.3390/ai3020019 · Cited by 81
A Systematic Review of Adversarial Machine Learning Attacks, Defensive Controls and Technologies
Malik, J., Muthalagu, R., & Pawar, P. M. (2024) · IEEE Access · doi:10.1109/access.2024.3423323 · Cited by 76
Metadata
- Authors
- Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A.
- Affiliations
- University of Monastir (TN)
- Corresponding author
- Bayoudh, Khaled
- Venue
- The Visual Computer
- Published
- 2021-06-10
- Volume
- 38
- Issue
- 8
- Pages
- 2939-2970
- ISSN
- 0178-2789 (ISSN-L)
- 1432-2315
- PMC
- PMC8192112
- OpenAlex
- W3168463823
- Semantic Scholar
- 2a9b33f66ccc3806af58bdab2319559f4f9d2c5e
- Cited by
- 456
- References
- 280
- Citations by year
- 2026: 70 · 2025: 148 · 2024: 119 · 2023: 70 · 2022: 42 · 2021: 7
- FWCI
- 21.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- The Visual Computer (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multimodal fusion, multimodal representation, multi-task learning, multimodal alignment, zero-shot learning, multimodal dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
