The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index; none of the 109 sources its claims cite is checked for support yet.
A Survey of Multimodal Sarcasm Detection
Open accessAbstract
Sarcasm is a rhetorical device that is used to convey the opposite of the literal meaning of an utterance. Sarcasm is widely used on social media and other forms of computer-mediated communication motivating the use of computational models to identify it automatically. While the clear majority of approaches to sarcasm detection have been carried out on text only, sarcasm detection often requires additional information present in tonality, facial expression, and contextual images. This has led to the introduction of multimodal models, opening the possibility to detect sarcasm in multiple modalities such as audio, images, text, and video. In this paper, we present the first comprehensive survey on multimodal sarcasm detection - henceforth MSD - to date. We survey papers published between 2018 and 2023 on the topic, and discuss the models and datasets used for this task. We also present future research directions in MSD.
Overview
CoChat Summary
## TL;DR
This paper is a survey of multimodal sarcasm detection (MSD), covering work published from 2018 to 2023 and summarizing datasets, methods, and future directions. It argues that multimodal cues such as images, audio, video, facial expression, and context are often necessary because text alone can be insufficient for sarcasm detection.
## Key findings
- The authors state this is the first comprehensive survey of MSD and that they survey over 60 papers.
- For visuo-textual MSD, the paper reports that the MMSD dataset has 19,818 training, 2,410 validation, and 2,409 test examples, with 10,560 sarcastic and 14,075 non-sarcastic samples; MMSD2.0 has 9,572 sarcastic and 10,240 non-sarcastic train samples, with validation/test sarcastic counts of 1,042 and 1,037 and non-sarcastic counts of 1,368 and 1,372.
- For audio-visual and textual MSD, the paper reports MUStARD contains 690 samples and is balanced; later variants include MUStARD++ with 601 sarcastic utterances, MUStARD++ Balanced with 691 sarcastic and 674 non-sarcastic samples, and SEEmoji MUStARD with 691 sarcastic and 601 non-sarcastic samples.
- The paper reports benchmark results showing performance improvements over time on MMSD and MUStARD-family datasets, for example Ding et al. (2022) at 93.85 accuracy / 94.06 F1 on MMSD, Qin et al. (2023) at 85.64 accuracy / 84.10 F1 on MMSD2.0, and Chauhan et al. (2022) at 77.9 precision / 76.7 F1 on SEEmoji-MUStARD.
## Methods
- This is a survey paper, not an original experimental study; it summarizes prior work on MSD rather than proposing a new model.
- The authors say they searched Google Scholar and databases including ACM Digital Library, IEEE Xplore, ACL Anthology, Springer, and CVF Open Access, using keywords such as multimodal, sarcasm detection, and related variants.
- They organize the literature into two main categories: (1) visuo-textual sarcasm detection and (2) audio-visual and textual sarcasm detection, and discuss datasets and methods in each.
## Limitations
- The paper states that MSD work is heavily concentrated in English, leaving a gap for other languages; it cites only a notable Spanish exception.
- It notes that sarcastic annotation is highly subjective, and that ground truth can depend on a single annotator or aggregation across annotators, motivating perspectivism.
- Evident from the text: because this is a survey, its conclusions depend on the scope of papers found through the authors’ search strategy and on the datasets and results reported by prior studies, not on a single controlled experimental evaluation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 6 Not valid · 98 Not checked · 15 Own results and interpretations
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While the clear majority of approaches to sarcasm detection have been carried out on text only”
- Also stated in: § 2 Textual vs. Multimodal Sarcasm Detection · boosted
“A clear majority of the previous works in automatic sarcasm detection have focused on text classification.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
- Rests on an assumption not shown here.
- Also stated in: § 2 Textual vs. Multimodal Sarcasm Detection · boosted
In the paper
“Cai et al. [2019] pointed out that identifying sarcasm solely based on text is sometimes impossible.”
What the sources say
- Not checked yetcites Cai et al., 2019
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Schifanella et al. [2016] presented one of the first datasets comprised of text and image pairs collected from user posts on Instagram, Twitter, and Tumblr.”
What the sources say
- Not checked yetcites Schifanella et al., 2016
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Sangwan et al. [2020] release a similar dataset composed of image and text pairs collected from Instagram.”
What the sources say
- Not checked yetcites Sangwan et al., 2020
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Das and Clark [2018a] introduced a partially multimodal sarcasm detection dataset collected from Facebook.”
What the sources say
- Not checked yetcites Das and Clark, 2018a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
21 more claims
Create a free account to see all 119 claims and to verify the 109 references still open.
- Probably not a substantial claim. 81
- It may state two separate claims as one. 30
- The quote only partly states this claim. 1
Citations
Cites
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00636 · Cited by 4,948
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020) · arXiv · doi:10.48550/arxiv.2006.11477 · Cited by 2,369
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations
Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., & Mihalcea, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1050 · Cited by 1,416 · cited 1×
“For non-sarcastic utterances, the authors reuse data from a multimodal emotion recognition dataset called MELD [Poria et al. , 2019].”
MethodVisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198
Multi-Modal Sarcasm Detection in Twitter with Hierarchical Fusion Model
Cai, Y., Cai, H., & Wan, X. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1239 · Cited by 377
Who cares about Sarcastic Tweets? Investigating the Impact of Sarcasm on Sentiment Analysis.
Maynard, D., & Greenwood, M. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2ios4bdd5xas · Cited by 349
Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)
Castro, S., Hazarika, D., Pérez-Rosas, V., Zimmermann, R., Mihalcea, R., & Poria, S. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1455 · Cited by 243
Detecting Sarcasm in Multimodal Social Platforms
Schifanella, R., de Juan, P., Tetreault, J., & Cao, L. (2016) · Proceedings of the 24th ACM international conference on Multimedia · doi:10.1145/2964284.2964321 · Cited by 237
Research works printed but not linked: 6
- [Alnajjar and H¨ am¨ al¨ ainen, 2021] Khalid Alnajjar and Mika H¨ am¨ al¨ ainen. ¡Qu´ e maravilla! multimodal sarcasm detection in Spanish: a dataset and a baseline. In NAACL , 2021.
- [Eyben et al. , 2010] Florian Eyben, Martin W¨ ollmer, and Bj¨ orn Schuller. Opensmile: The munich versatile and fast open-source audio feature extractor. In ACM MM , 2010.
- [Gu et al. , 2018] Yue Gu, Kangning Yang, Shiyu Fu, Shuhong Chen, Xinyu Li, and Ivan Marsic. Hybrid attention based multimodal network for spoken language classification. In ACL , 2018. - [Hazarika et al. , 2018] Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, and Rada Mihalcea. CASCADE: Contextual sarcasm detection in online discussion forums. In COLING , 2018.
- [Lu et al. , 2019] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for v-l tasks. In NeurIPS , 2019.
- [Oprea and Magdy, 2020] Silviu Oprea and Walid Magdy. iSarcasm: A dataset of intended sarcasm. In ACL , 2020.
- [Radford et al. , 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML , 2021.
Cited by
Still being checked.
Metadata
- Authors
- Farabi, S., Ranasinghe, T., Kanojia, D., Kong, Y., & Zampieri, M.
- Affiliations
- George Mason University (US)
- Lancaster University (GB)
- University of Surrey (GB)
- Michigan State University (US)
- Corresponding author
- Farabi, Shafkat
- Venue
- Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence
- Published
- 2024-08
- Pages
- 8020-8028
- OpenAlex
- W4401023717
- Cited by
- 17
- References
- 47
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 1
- FWCI
- 4.91
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- other-oa
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion, other-oa)
- Lancaster EPrints (Lancaster University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- sarcasm detection, multimodal sarcasm detection, multimodal models, social media, computer-mediated communication, audio, facial expressions, datasets
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Survey of Multimodal Sarcasm Detection. https://app.cochat.ai/papers/10.24963/ijcai.2024/887
- The paper
- Farabi, S., Ranasinghe, T., Kanojia, D., Kong, Y., & Zampieri, M. (2024). A Survey of Multimodal Sarcasm Detection. Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence. https://doi.org/10.24963/ijcai.2024/887
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.