CoChat

Multiscale Feature Extraction and Fusion of Image and Text in VQA

Open access
Lu, Siyu, Ding, Yueming, Liu, Mingzhe, Yin, Zhengtong, Yin, Lirong, Zheng, Wenfeng · 2023 · International Journal of Computational Intelligence Systems

The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.

Abstract

The Visual Question Answering (VQA) system is the process of finding useful information from images related to the question to answer the question correctly. It can be widely used in the fields of visual assistance, automated security surveillance, and intelligent interaction between robots and humans. However, the accuracy of VQA has not been ideal, and the main difficulty in its research is that the image features cannot well represent the scene and object information, and the text information cannot be fully represented. This paper used multi-scale feature extraction and fusion methods in the image feature characterization and text information representation sections of the VQA system, respectively to improve its accuracy. Firstly, aiming at the image feature representation problem, multi-scale feature extraction and fusion method were adopted, and the image features output of different network layers were extracted by a pre-trained deep neural network, and the optimal scheme of feature fusion method was found through experiments. Secondly, for the representation of sentences, a multi-scale feature method was introduced to characterize and fuse the word-level, phrase-level, and sentence-level features of sentences. Finally, the VQA model was improved using the multi-scale feature extraction and fusion method. The results show that the addition of multi-scale feature extraction and fusion improves the accuracy of the VQA model.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s44196-023-00233-6

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 40 references

Research works printed but not linked: 4

  1. Ren, M., Kiros, R., Zemel, R.: Exploring models and data for image question answering. Adv. Neural. Inf. Process. Syst. 28, 2953–2961 (2015)
  2. Auer, S., et al. DBpedia: A nucleus for a web of open data. in 6th International Semantic Web Conference/2nd Asian Semantic Web Conference (ISWC 2007/ASWC 2007). Busan, South Korea: Springer-Verlag Berlin. (2007)
  3. Adelson, E.H., et al.: Pyramid methods in image processing. RCA Eng 29(6), 33–41 (1984)
  4. Ben-younes, H., et al. MUTAN: Multimodal Tucker Fusion for Visual Question Answering. In 16th IEEE International Conference on Computer Vision (ICCV). Venice, ITALY: Ieee. (2017) · DOI · Paper page

Cited by

See all 429

Metadata

Authors
Lu, S., Ding, Y., Liu, M., Yin, Z., Yin, L., & Zheng, W.
Affiliations
  • University of Electronic Science and Technology of China (CN)
  • Wenzhou University of Technology (CN)
  • Guizhou University (CN)
  • Louisiana State University (US)
Corresponding author
  • Liu, Mingzhe
  • Zheng, Wenfeng
Venue
International Journal of Computational Intelligence Systems
Published
2023-04-11
Volume
16
Issue
1
ISSN
1875-6883 (ISSN-L)
OpenAlex
W4364377616
Cited by
422
References
41
Citations by year
2026: 12 · 2025: 44 · 2024: 66 · 2023: 114
FWCI
21.44
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Computational Intelligence Systems (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Sichuan Science and Technology Program (2021YFQ0003)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
Keywords
visual question answering, multi-scale feature extraction, feature fusion, image-text fusion, visual feature representation, sentence representation, word-level features, sentence-level features, pre-trained deep neural networks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Multiscale Feature Extraction and Fusion of Image and Text in VQA. https://app.cochat.ai/papers/10.1007/s44196-023-00233-6
The paper
Lu, S., Ding, Y., Liu, M., Yin, Z., Yin, L., & Zheng, W. (2023). Multiscale Feature Extraction and Fusion of Image and Text in VQA. International Journal of Computational Intelligence Systems. https://doi.org/10.1007/s44196-023-00233-6

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.