CoChat

PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text

Wang, Wenhai, Xie, Enze, Li, Xiang, Liu, Xuebo, Liang, Ding, Zhibo, Yang, Lu, Tong, Shen, Chunhua · 2021 · IEEE Transactions on Pattern Analysis and Machine Intelligence

The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.

Abstract

Scene text detection and recognition have been well explored in the past few years. Despite the progress, efficient and accurate end-to-end spotting of arbitrarily-shaped text remains challenging. In this work, we propose an end-to-end text spotting framework, termed PAN++, which can efficiently detect and recognize text of arbitrary shapes in natural scenes. PAN++ is based on the kernel representation that reformulates a text line as a text kernel (central region) surrounded by peripheral pixels. By systematically comparing with existing scene text representations, we show that our kernel representation can not only describe arbitrarily-shaped text but also well distinguish adjacent text. Moreover, as a pixel-based representation, the kernel representation can be predicted by a single fully convolutional network, which is very friendly to real-time applications. Taking the advantages of the kernel representation, we design a series of components as follows: 1) a computationally efficient feature enhancement network composed of stacked Feature Pyramid Enhancement Modules (FPEMs); 2) a lightweight detection head cooperating with Pixel Aggregation (PA); and 3) an efficient attention-based recognition head with Masked RoI. Benefiting from the kernel representation and the tailored components, our method achieves high inference speed while maintaining competitive accuracy. Extensive experiments show the superiority of our method. For example, the proposed PAN++ achieves an end-to-end text spotting F-measure of 64.9 at 29.2 FPS on the Total-Text dataset, which significantly outperforms the previous best method. Code will be available at: git.io/PAN.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpami.2021.3077555

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 77 references

Research works printed but not linked: 1

  1. [58] C. Yao, X. Bai, W. Liu, Y. Ma, and Z. Tu, 'Detecting texts of arbitrary orientations in natural images,' in Proc. IEEE Conf. Comp. Vis. Patt. Recogn. , 2012. - · DOI · Paper page

Cited by

Items 101–162 of 162

Metadata

Authors
Wang, W., Xie, E., Li, X., Liu, X., Liang, D., Zhibo, Y., Lu, T., & Shen, C.
Affiliations
  • Nanjing University (CN)
  • University of Hong Kong (HK)
  • Nanjing University of Science and Technology (CN)
  • Sensetime (China) (CN)
  • Monash University (AU)
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2021
Volume
44
Issue
9
Pages
1-1
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
OpenAlex
W3159307593
Cited by
140
References
73
Citations by year
2026: 6 · 2025: 31 · 2024: 47 · 2023: 27 · 2022: 26 · 2021: 3
FWCI
9.26
Type
journal article
Language
en
Open access
closed
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Vehicle License Plate Recognition
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
scene text detection, scene text recognition, lightweight detection head, pixel aggregation, real-time inference
MeSH terms
Algorithms

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.