Cited by 36Advanced Image and Video Retrieval TechniquesImage Retrieval and Classification TechniquesAdvanced Neural Network Applications

Attention Weighted Local Descriptors

Wang, Changwei, Xu, Rongtao, Lu, Ke, Xu, Shibiao, Meng, Weiliang, Zhang, Yuyang, Fan, Bin, Zhang, Xiaopeng·2023·IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

Local features detection and description are widely used in many vision applications with high industrial and commercial demands. With large-scale applications, these tasks raise high expectations for both the accuracy and speed of local features. Most existing studies on local features learning focus on the local descriptions of individual keypoints, which neglect their relationships established from global spatial awareness. In this paper, we present AWDesc with a consistent attention mechanism (CoAM) that opens up the possibility for local descriptors to embrace image-level spatial awareness in both the training and matching stages. For local features detection, we adopt local features detection with feature pyramid to obtain more stable and accurate keypoints localization. For local features description, we provide two versions of AWDesc to cope with different accuracy and speed requirements. On the one hand, we introduce Context Augmentation to address the inherent locality of convolutional neural networks by injecting non-local context information, so that local descriptors can "look wider to describe better". Specifically, well-designed Adaptive Global Context Augmented Module (AGCA) and Diverse Surrounding Context Augmented Module (DSCA) are proposed to construct robust local descriptors with context information from global to surrounding. On the other hand, we design an extremely lightweight backbone network coupled with the proposed special knowledge distillation strategy to achieve the best trade-off in accuracy and speed. What is more, we perform thorough experiments on image matching, homography estimation, visual localization, and 3D reconstruction tasks, and the results demonstrate that our method surpasses the current state-of-the-art local descriptors. Code is available at: https://github.com/vignywang/AWDesc.

Integrity signals

Registry

Crossref · 10.1109/tpami.2023.3266728

Cites

80 of 80 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • NeRF and Semantic-NeRF encode scene geometry and semantics into compact neural fields, enabling robust localization in new environments and showing multimodal fusion potential for perception and mapping.

    “Neural implicit representations, like NeRF [135] and Semantic-NeRF [136], encode scene geometry and semantics into compact neural fields, enabling robust localization in new environments and showcasing the potential of multimodal fusion for bridging perception and mapping.”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Wang, C., Xu, R., Lu, K., Xu, S., Meng, W., Zhang, Y., Fan, B., & Zhang, X.
Affiliations
  • Chinese Academy of Sciences (CN)
  • School of Engineering Science, University of Chinese Academy of Sciences, Beijing, China
  • Beijing University of Posts and Telecommunications (CN)
  • Thunder Software Technology Company, Ltd., Chengdu, China
  • University of Science and Technology Beijing (CN)
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2023-09-01
Volume
45
Issue
9
Pages
10632-10649
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
OpenAlex
W4366351734
Cited by
36
References
80
Citations by year
2026: 6 · 2025: 18 · 2024: 10 · 2023: 2
FWCI
3.26
Type
journal article
Language
en
Open access
closed
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • National Natural Science Foundation of China (U21A20515, 62032022, 62271074, 62222302, U2003109, 62171321, 62071157, 62162044)
  • Open Research Fund of Key Laboratory of Space Utilization, Chinese Academy of Sciences (LSU-KFJJ-2021-05)
  • National Laboratory of Pattern Recognition
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
local descriptors, local feature detection, attention mechanism, spatial awareness, context augmentation, feature pyramid, lightweight backbone, knowledge distillation, image matching, homography estimation, visual localization, 3D reconstruction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review