PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text
The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Abstract
Scene text detection and recognition have been well explored in the past few years. Despite the progress, efficient and accurate end-to-end spotting of arbitrarily-shaped text remains challenging. In this work, we propose an end-to-end text spotting framework, termed PAN++, which can efficiently detect and recognize text of arbitrary shapes in natural scenes. PAN++ is based on the kernel representation that reformulates a text line as a text kernel (central region) surrounded by peripheral pixels. By systematically comparing with existing scene text representations, we show that our kernel representation can not only describe arbitrarily-shaped text but also well distinguish adjacent text. Moreover, as a pixel-based representation, the kernel representation can be predicted by a single fully convolutional network, which is very friendly to real-time applications. Taking the advantages of the kernel representation, we design a series of components as follows: 1) a computationally efficient feature enhancement network composed of stacked Feature Pyramid Enhancement Modules (FPEMs); 2) a lightweight detection head cooperating with Pixel Aggregation (PA); and 3) an efficient attention-based recognition head with Masked RoI. Benefiting from the kernel representation and the tailored components, our method achieves high inference speed while maintaining competitive accuracy. Extensive experiments show the superiority of our method. For example, the proposed PAN++ achieves an end-to-end text spotting F-measure of 64.9 at 29.2 FPS on the Total-Text dataset, which significantly outperforms the previous best method. Code will be available at: git.io/PAN.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 13×
“In 13 these experiments, we use ResNet18 as the default backbone network.”
MethodInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“In virtue of the powerful representation of deep neural networks [1], [2], scene text detection and recognition has witnessed great progress in the past a few years [3], [4], [5], [6], [7], [8].”
BackgroundHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 12×
“At time step 0 (initial time step), the decoder takes SOS’s feature vector fs and zero LSTM initial states as input.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“All models are optimized by using ADAM [66] optimizer with a batch size of 16 on 4 GPUs.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 4×
“3(f), where an attention-based text recognition head was added on Mask R-CNN to simultaneously detect and recognize irregular text lines.”
MethodInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“In these experiments, we use the ResNet [2] pre-trained on ImageNet [65] as the backbone network of our method.”
MethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
““Conv” and “BN” represent regular convolution [49] and Batch Normalization [50], respectively.”
BackgroundFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · cited 8×
“Second, we compare our feature enhancement network (2- stacked FPEMs) with other widely used methods, including FPN [51], PAFPN [74], and NAS-FPN [75].”
BackgroundMethodInfluentialHe, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
““Conv” and “BN” represent regular convolution [49] and Batch Normalization [50], respectively.”
BackgroundMobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015 · cited 2×
““DWConv”, “Conv” and “BN” represent, depthwise convolution [48], regular convolution [49] and Batch Normalization [50], respectively.”
BackgroundMethodSelf-citationZhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · cited 1×
“to [67], we use the “poly” learning rate strategy in which the initial rate is multiplied by (1 − iter max iter ) , and the power is set to 0.”
MethodV-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931 · cited 1×
“Considering the extreme imbalance of text and non-text pixels, we follow [4], [17] and adopt dice loss [53] to optimize the segmentation result of text regions Ptex and of text kernels Pker .”
MethodIdentity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665 · cited 12×
“In 13 these experiments, we use ResNet18 as the default backbone network.”
BackgroundMethodInfluentialPath Aggregation Network for Instance Segmentation
Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00913 · Cited by 10,011
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Shi, B., Bai, X., & Yao, C. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2646371 · Cited by 3,172
Training Region-Based Object Detectors with Online Hard Example Mining
Shrivastava, A., Gupta, A., & Girshick, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.89 · Cited by 2,686
NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection
Ghiasi, G., Lin, T.-Y., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00720 · Cited by 2,099 · cited 4×
“%) and close to NAS-FPN [75] (66.9%).”
MethodInfluentialEAST: An Efficient and Accurate Scene Text Detector
Zhou, X., Yao, C., Wen, H., Wang, Y., Zhou, S., He, W., & Liang, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.283 · Cited by 1,850
Synthetic Data for Text Localisation in Natural Images
Gupta, A., Vedaldi, A., & Zisserman, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.254 · Cited by 1,563 · cited 1×
“SynthText [62] is a large-scale synthetically generated dataset containing 800K synthetic images.”
BackgroundICDAR 2015 competition on Robust Reading
Karatzas, D., Gomez-Bigorda, L., Nicolaou, A., Ghosh, S., Bagdanov, A., Iwamura, M., Matas, J., Neumann, L., Chandrasekhar, V. R., Lu, S., et al. (2015) · 2015 13th International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2015.7333942 · Cited by 1,547
Arbitrary-Oriented Scene Text Detection via Rotation Proposals
Ma, J., Shao, W., Ye, H., Wang, L., Wang, H., Zheng, Y., & Xue, X. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2818020 · Cited by 1,260 · cited 1×
“In the text detection task, to make fair comparisons, we train PAN++ (without the recognition head) with two widely used strategies as follows: 1) following [3], [17], [68] to train models without external text datasets; 2) following [4], [17], [34] to finetune models pre-trained on external text dataset (e.”
MethodCharacter Region Awareness for Text Detection
Baek, Y., Lee, B., Han, D., Yun, S., & Lee, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00959 · Cited by 1,103 · cited 1×
“4 CRAFT [56] L: 1280 VGG16 SynthText 87.”
Detecting Text in Natural Image with Connectionist Text Proposal Network
Tian, Z., Huang, W., He, T., He, P., & Qiao, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_4 · Cited by 1,053 · cited 1×
“For most existing text detectors [3], [4], [12], a convolutional neural network is first used to generate feature maps of an input image, and then rectangular or polygonal bounding boxes of scene texts are generated using a decoder.”
MethodTextBoxes: A Fast Text Detector with a Single Deep Neural Network
Liao, M., Shi, B., Bai, X., Wang, X., & Liu, W. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11196 · Cited by 955 · cited 2×
“TextBoxes [33] MS ResNet50 Jointly Train SynthText, IC13, IC15 36.”
MethodASTER: An Attentional Scene Text Recognizer with Flexible Rectification
Shi, B., Yang, M., Wang, X., Lyu, P., Yao, C., & Bai, X. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2848939 · Cited by 906 · cited 5×
“On the other side, text recognition methods [5], [6], [13] often conduct a sequential prediction network on top of image patches of text lines.”
BackgroundMethodInfluentialTextBoxes++: A Single-Shot Oriented Scene Text Detector
Liao, M., Shi, B., & Bai, X. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2825107 · Cited by 897 · cited 3×
“(a) The quadrilateral representation [3], [7], [9] fail to locate curved text lines.”
BackgroundMethodShape Robust Text Detection With Progressive Scale Expansion Network
Wang, W., Xie, E., Li, X., Hou, W., Lu, T., Yu, G., & Shao, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00956 · Cited by 860 · cited 13×
“Compared with our conference versions PSENet [4] and PAN [17], the major extension of PAN++ lies in the text recognition module and the end-to-end text spotting framework.”
BackgroundMethodInfluentialSelf-citationDetecting Oriented Text in Natural Images by Linking Segments
Shi, B., Bai, X., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.371 · Cited by 775
Detecting texts of arbitrary orientations in natural images
Yao, C., Bai, X., Liu, W., Ma, Y., & Tu, Z. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247787 · Cited by 765
Deep Features for Text Spotting
Jaderberg, M., Vedaldi, A., & Zisserman, A. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_34 · Cited by 653 · cited 1×
“Character-based text recognizers [40], [41] mostly first detect individual characters and then group them into words.”
BackgroundTextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes
Long, S., Ruan, J., Zhang, W., He, X., Wu, W., & Yao, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_2 · Cited by 636
PixelLink: Detecting Scene Text via Instance Segmentation
Deng, D., Liu, H., Li, X., & Cai, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12269 · Cited by 632
Self-citationEfficient and Accurate Arbitrary-Shaped Text Detection With Pixel Aggregation Network
Wang, W., Xie, E., Song, X., Zang, Y., Wang, W., Lu, T., Yu, G., & Shen, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00853 · Cited by 632 · cited 9×
“Compared with our conference versions PSENet [4] and PAN [17], the major extension of PAN++ lies in the text recognition module and the end-to-end text spotting framework.”
BackgroundMethodInfluentialSelf-citationTotal-Text: A Comprehensive Dataset for Scene Text Detection and Recognition
Ch'ng, C. K., & Chan, C. S. (2017) · 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2017.157 · Cited by 571 · cited 4×
“Qualitative text spotting results of PAN++ on Total-Text [23], IC15 [25], and RCTW-17 [61].”
BackgroundMethodInfluentialRotation-Sensitive Regression for Oriented Scene Text Detection
Liao, M., Zhu, Z., Shi, B., Xia, G.-S., & Bai, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00619 · Cited by 563
FOTS: Fast Oriented Text Spotting with a Unified Network
Liu, X., Liang, D., Yan, S., Chen, D., Qiao, Y., & Yan, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00595 · Cited by 554
Self-citationMask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes
Lyu, P., Liao, M., Yao, C., Wu, W., & Bai, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_5 · Cited by 522
PhotoOCR: Reading Text in Uncontrolled Conditions
Bissacco, A., Cummins, M., Netzer, Y., & Neven, H. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.102 · Cited by 457 · cited 1×
“Character-based text recognizers [40], [41] mostly first detect individual characters and then group them into words.”
BackgroundABCNet: Real-Time Scene Text Spotting With Adaptive Bezier-Curve Network
Liu, Y., Chen, H., Shen, C., He, T., Jin, L., & Wang, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00983 · Cited by 457
Self-citationNayef, N., Yin, F., Bizid, I., Choi, H., Feng, Y., Karatzas, D., Luo, Z., Pal, U., Rigaud, C., Chazalon, J., et al. (2017) · 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2017.237 · Cited by 455 · cited 4×
“Specifically, when using the finetuning strategy, we first pre-train models on the training images in the SynthText, COCO-Text, and IC17-MLT datasets for 150K iterations, and then we fine-tune the pre-trained models on TotalText or CTW1500 for 7K iterations.”
BackgroundInfluentialA generic solution to polygon clipping
Vatti, B. R. (1992) · Communications of the ACM · doi:10.1145/129902.129906 · Cited by 450
Deep Direct Regression for Multi-oriented Scene Text Detection
He, W., Zhang, X.-Y., Yin, F., & Liu, C.-L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.87 · Cited by 432
Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition
Li, H., Wang, P., Shen, C., & Zhang, G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018610 · Cited by 432
Self-citationMulti-oriented Scene Text Detection via Corner Localization and Region Segmentation
Lyu, P., Yao, C., Wu, W., Yan, S., & Bai, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00788 · Cited by 389
TextField: Learning a Deep Direction Field for Irregular Scene Text Detection
Xu, Y., Wang, Y., Zhou, W., Wang, Y., Yang, Z., & Bai, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2900589 · Cited by 345 · cited 1×
“6 TextField [55] 768×768 VGG16 SynthText 81.”
Single Shot Text Detector with Regional Attention
He, P., Huang, W., He, T., Zhu, Q., Qiao, Y., & Li, X. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.331 · Cited by 344 · cited 1×
“9 SSTD [71] 704×704 VGG16 Private Data 80.”
Self-citationA Unified Framework for Multioriented Text Detection and Recognition
Yao, C., Bai, X., & Liu, W. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2353813 · Cited by 343 · cited 1×
“Because its training set is small, we follow the previous works [3], [37], [59] to include the 400 images of HUST-TR400 [60] as training data.”
MethodLook More Than Once: An Accurate Detector for Text of Arbitrary Shapes
Zhang, C., Liang, B., Huang, Z., En, M., Han, J., Ding, E., & Ding, X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01080 · Cited by 302 · cited 3×
“1 LOMO [57] L: 1536 ResNet50 SynthText 91.”
BackgroundReading Scene Text in Deep Convolutional Sequences
He, P., Huang, W., Qiao, Y., Loy, C., & Tang, X. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10465 · Cited by 300 · cited 1×
“CTC-based methods [5], [42], [43], [44] often stack RNN on top of CNN to capture long-range sequence features.”
MethodESIR: End-To-End Scene Text Recognition via Iterative Image Rectification
Zhan, F., & Lu, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00216 · Cited by 296 · cited 1×
“[47] used a spatial rectification network to rectify irregularly-shaped text lines into straight ones before recognition.”
MethodAON: Towards Arbitrarily-Oriented Text Recognition
Cheng, Z., Xu, Y., Bai, F., Niu, Y., Pu, S., & Zhou, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00584 · Cited by 292 · cited 1×
“In attentionbased methods [6], [13], [45], [46], RNN or Transformer [21] is used to predict a character at each step based on the features from attention mechanisms and previous steps.”
MethodDeep TextSpotter: An End-to-End Trainable Scene Text Localization and Recognition Framework
Busta, M., Neumann, L., & Matas, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.242 · Cited by 244
Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes
Liao, M., Lyu, P., He, M., Yao, C., Wu, W., & Bai, X. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2937086 · Cited by 244 · cited 5×
“The improved version [30] of Mask TextSpotter removed the dependence of character-level annotations.”
BackgroundMethodInfluentialScene Text Detection with Supervised Pyramid Context Network
Xie, E., Zang, Y., Shao, S., Yu, G., Yao, C., & Li, G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019038 · Cited by 236
Self-citationAn End-to-End TextSpotter with Explicit Alignment and Attention
He, T., Tian, Z., Huang, W., Shen, C., Qiao, Y., & Sun, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00527 · Cited by 234
Self-citationTowards End-to-End Text Spotting with Convolutional Recurrent Neural Networks
Li, H., Wang, P., & Shen, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.560 · Cited by 230 · cited 4×
“Last, the efficiency of existing end-to-end text spotters [7], [8], [14], [15], [16] remains insufficient for real-world applications.”
BackgroundMethodInfluentialSelf-citationTextDragon: An End-to-End Framework for Arbitrary Shaped Text Spotting
Feng, W., He, W., Yin, F., Zhang, X.-Y., & Liu, C.-L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00917 · Cited by 226 · cited 6×
“5 TextDragon [32] - VGG16 Finetune SynthText, Total-Text 82.”
MethodInfluentialICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17)
Shi, B., Yao, C., Liao, M., Yang, M., Xu, P., Cui, L., Belongie, S., Lu, S., & Bai, X. (2017) · 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2017.233 · Cited by 224 · cited 9×
“We choose the RCTW-17 [61] dataset to test the effectiveness of our method in detecting and recognizing Chinese text, which is a commonly used language that is completely different from English.”
BackgroundMethodInfluentialAccurate Scene Text Recognition Based on Recurrent Neural Network
Su, B., & Lu, S. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16865-4_3 · Cited by 221 · cited 1×
“CTC-based methods [5], [42], [43], [44] often stack RNN on top of CNN to capture long-range sequence features.”
MethodText Flow: A Unified Text Detection System in Natural Scene Images
Tian, S., Pan, Y., Huang, C., Lu, S., Yu, K., & Tan, C. L. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.528 · Cited by 220 · cited 1×
“To detect curved text lines, some early works [34], [35] followed a bottom-up framework to first detect characters or text fragments and then link them.”
BackgroundCOCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images
Veit, A., Matera, T., Neumann, L., Matas, J., & Belongie, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.07140 · Cited by 191 · cited 1×
“COCO-Text [63] contains 63,686 images and three versions of the annotations (V1.”
BackgroundDetecting Curve Text in the Wild: New Dataset and New Solution
Yuliang, L., Jin, L., Zhang, S., & Zhang, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.02170 · Cited by 189 · cited 5×
“4 points higher than ABCNet [24], while its inference speed is faster.”
BackgroundMethodInfluentialLearning to Read Irregular Text with Attention Mechanisms
Yang, X., He, D., Zhou, Z., Kifer, D., & Giles, C. L. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/458 · Cited by 183
Convolutional Character Networks
Xing, L., Tian, Z., Huang, W., & Scott, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00922 · Cited by 181 · cited 5×
“Most methods [8], [16], [30], [31], [32] mentioned in Sec.”
BackgroundMethodInfluentialSu, B., & Lu, S. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.10.016 · Cited by 157 · cited 1×
“CTC-based methods [5], [42], [43], [44] often stack RNN on top of CNN to capture long-range sequence features.”
MethodMSR: Multi-Scale Shape Regression for Scene Text Detection
Xue, C., Lu, S., & Zhang, W. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/139 · Cited by 154
Learning Markov Clustering Networks for Scene Text Detection
Liu, Z., Lin, G., Yang, S. C., Feng, J., Lin, W., & Goh, W. L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00725 · Cited by 141
Towards Unconstrained End-to-End Text Spotting
Qin, S., Bissaco, A., Raptis, M., Fujii, Y., & Xiao, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00480 · Cited by 133
TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network
Sun, Y., Zhang, C., Huang, Z., Liu, J., Han, J., & Ding, E. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20893-6_6 · Cited by 76
ICDAR2017 Robust Reading Challenge on COCO-Text
Gómez, R., Shi, B., Gómez, L., Numann, L., Veit, A., Matas, J., Belongie, S., & Karatzas, D. (2017) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2017.234 · Cited by 59
Multi-Oriented Scene Text Detection via Corner Localization and Region SegmentationPreprint
Lyu, P., Yao, C., Wu, W., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08948 · Cited by 42 · cited 2×
“Following [34], [37], [59], this dataset is used for pre-training or jointly training models of PAN++.”
MethodA pooling based scene text proposal technique for scene text reading in the wild
NguyenVan, D., Lu, S., Tian, S., Ouarti, N., & Mokhtari, M. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.10.012 · Cited by 21 · cited 1×
“[29] developed an end-to-end text reading framework by incorporating a pooling-based scene text proposal technique, where false alarms elimination and words recognition are performed simultaneously.”
Background
Research works printed but not linked: 1
- [58] C. Yao, X. Bai, W. Liu, Y. Ma, and Z. Tu, 'Detecting texts of arbitrary orientations in natural images,' in Proc. IEEE Conf. Comp. Vis. Patt. Recogn. , 2012. - · DOI · Paper page
Cited by
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259
Self-citationMPDIoU: A Loss for Efficient and Accurate Bounding Box RegressionPreprint
Ma, S., & Xu, Y. D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.07662 · Cited by 832
On the Arbitrary-Oriented Object Detection: Classification Based Approaches Revisited
Yang, X., & Yan, J. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01593-w · Cited by 195
“SBD [20], SegLink [89], TextField [62], DBNet [33], PAN++ [64], PolarMask++ [65]), model ensemble (e.”
BackgroundSwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition
Huang, M., Liu, Y., Peng, Z., Liu, C., Lin, D., Zhu, S., Yuan, N., Ding, K., & Jin, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00455 · Cited by 167
DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting
Ye, M., Zhang, J., Zhao, S., Liu, J., Liu, T., Du, B., & Tao, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01854 · Cited by 142
Metadata
- Authors
- Wang, W., Xie, E., Li, X., Liu, X., Liang, D., Zhibo, Y., Lu, T., & Shen, C.
- Affiliations
- Nanjing University (CN)
- University of Hong Kong (HK)
- Nanjing University of Science and Technology (CN)
- Sensetime (China) (CN)
- Monash University (AU)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021
- Volume
- 44
- Issue
- 9
- Pages
- 1-1
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3159307593
- Semantic Scholar
- 70aa3857040d2934ab7544988468ede697cb90c9
- Cited by
- 140
- References
- 73
- Citations by year
- 2026: 6 · 2025: 31 · 2024: 47 · 2023: 27 · 2022: 26 · 2021: 3
- FWCI
- 9.26
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Vehicle License Plate Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- scene text detection, scene text recognition, lightweight detection head, pixel aggregation, real-time inference
- MeSH terms
- Algorithms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.