The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Improving Multimodal Named Entity Recognition via Entity Span Detection with Unified Multimodal Transformer
Open accessAbstract
In this paper, we study Multimodal Named Entity Recognition (MNER) for social media posts.Existing approaches for MNER mainly suffer from two drawbacks: (1) despite generating word-aware visual representations, their word representations are insensitive to the visual context; (2) most of them ignore the bias brought by the visual context.To tackle the first issue, we propose a multimodal interaction module to obtain both image-aware word representations and word-aware visual representations.To alleviate the visual bias, we further propose to leverage purely text-based entity span detection as an auxiliary module, and design a Unified Multimodal Transformer to guide the final predictions with the entity span predictions.Experiments show that our unified approach achieves the new state-of-the-art performance on two benchmark datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Neural Architectures for Named Entity Recognition
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1030 · Cited by 4,401
Incorporating non-local information into information extraction systems by Gibbs sampling
Finkel, J. R., Grenager, T., & Manning, C. (2005) · Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL '05 · doi:10.3115/1219840.1219885 · Cited by 3,008
ImageNet: A large-scale hierarchical image database
Deng, J. H., Dong, W. F., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206848 · Cited by 2,714
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567
Multimodal Transformer for Unaligned Multimodal Language Sequences
Tsai, Y.-H. H., Bai, S., Liang, P. P., Kolter, J. Z., Morency, L.-P., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1656 · Cited by 2,259 · cited 1×
“2.b, to learn better word representations with the guidance of associated images, we first employ an m -head cross-modal attention mechanism (Tsai et al., 2019), by treating V ∈ R d × 49 as queries, and R ∈ R d × ( n +1) as keys and values: where CA i refers to the i -th head of cross-modal…”
MethodNamed Entity Recognition with Bidirectional LSTM-CNNs
Chiu, J. P. C., & Nichols, E. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00104 · Cited by 1,909
A Survey on Deep Learning for Named Entity Recognition
Li, J., Sun, A., Han, J., & Li, C. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2981314 · Cited by 1,567
Design challenges and misconceptions in named entity recognition
Ratinov, L., & Roth, D. (2009) · Proceedings of the Thirteenth Conference on Computational Natural Language Learning - CoNLL '09 · doi:10.3115/1596374.1596399 · Cited by 1,448
Morgan, M. C. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.17575540 · Cited by 1,283
Named Entity Recognition in Tweets: An Experimental Study
Ritter, A., Clark, S., & Etzioni, O. (2011) · Cited by 1,181
Named entity recognition using an HMM-based chunk tagger
Zhou, G., & Su, J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073163 · Cited by 715
Open domain event extraction from twitter
Ritter, A., Mausam, Etzioni, O., & Clark, S. (2012) · Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2339530.2339704 · Cited by 592
A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
Yadav, V., & Bethard, S. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11470 · Cited by 464
Sang, E. F. T. K., & Veenstra, J. (1999) · Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics - · doi:10.3115/977035.977059 · Cited by 456
Named entity recognition through classifier combination
Florian, R., Ittycheriah, A., Jing, H., & Zhang, T. (2003) · Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 - · doi:10.3115/1119176.1119201 · Cited by 399
Adaptive Co-attention Network for Named Entity Recognition in Tweets
Zhang, Q., Fu, J., Liu, X., & Huang, X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11962 · Cited by 381
Analysis of named entity recognition and linking for tweets
Derczynski, L., Maynard, D., Rizzo, G., van Erp, M., Gorrell, G., Troncy, R., Petrak, J., & Bontcheva, K. (2015) · Information Processing & Management · doi:10.1016/j.ipm.2014.10.006 · Cited by 340
Named entity recognition: a maximum entropy approach using global information
Chieu, H. L., & Ng, H. T. (2002) · Proceedings of the 19th international conference on Computational linguistics - · doi:10.3115/1072228.1072253 · Cited by 309
Visual Attention Model for Name Tagging in Multimodal Social Media
Lu, D., Neves, L., Carvalho, V., Zhang, N., & Ji, H. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1185 · Cited by 298
Lexicon Infused Phrase Embeddings for Named Entity Resolution
Passos, A., Kumar, V., & McCallum, A. (2015) · Cited by 277
Joint Entity Recognition and Disambiguation
Luo, G., Huang, X., Lin, C.-Y., & Nie, Z. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1104 · Cited by 248
TwiNER: named entity recognition in targeted twitter stream
Li, C., Weng, J., He, Q., Yao, Y., Datta, A., Sun, A., & Lee, B. (2012) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2348283.2348380 · Cited by 239
Phrase clustering for discriminative learning
Lin, D., & Wu, X. (2009) · Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - ACL-IJCNLP '09 · doi:10.3115/1690219.1690290 · Cited by 226
Multimodal Named Entity Recognition for Short Social Media Posts
Moon, S., Neves, L., & Carvalho, V. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1078 · Cited by 200
Baldwin, T., de Marneffe, M.-C., Han, B., Kim, Y.-B., Ritter, A., & Xu, W. (2015) · Proceedings of the Workshop on Noisy User-generated Text · doi:10.18653/v1/w15-4319 · Cited by 189
Design Challenges and Misconceptions in Neural Sequence LabelingPreprint
Yang, J., Liang, S., & Zhang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04470 · Cited by 177
Multi-channel BiLSTM-CRF Model for Emerging Named Entity Recognition in Social Media
Lin, B. Y., Xu, F. F., Luo, Z., & Zhu, K. Q. (2017) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/w17-4421 · Cited by 137
Bidirectional LSTM for Named Entity Recognition in Twitter Messages
Limsopatham, N., & Collier, N. (2016) · Apollo (University of Cambridge) · doi:10.17863/cam.7201 · Cited by 102
Tweet Segmentation and Its Application to Named Entity Recognition
Li, C., Sun, A., Weng, J., & He, Q. (2014) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2327042 · Cited by 97
(2015) · doi:10.4230/oasics.slate.2021.8 · Cited by 1
Metadata
- Authors
- Yu, J., Jiang, J., Yang, L., & Xia, R.
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 3342-3352
- OpenAlex
- W3035448883
- Cited by
- 281
- References
- 38
- FWCI
- 8.11
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.