Rosetta: Large Scale System for Text Detection and Recognition in Images
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
In this paper we present a deployed, scalable optical character recognition (OCR) system, which we call Rosetta , designed to process images uploaded daily at Facebook scale. Sharing of image content has become one of the primary ways to communicate information among internet users within social networks such as Facebook, and the understanding of such media, including its textual information, is of paramount importance to facilitate search and recommendation applications. We present modeling techniques for efficient detection and recognition of text in images and describe Rosetta 's system architecture. We perform extensive evaluation of presented technologies, explain useful practical approaches to build an OCR system at scale, and provide insightful intuitions as to why and how certain components work based on the lessons learnt during the development and deployment of the system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A novel OCR system is developed, named as Rosetta, wherein faster-RCNN is utilized for detecting the text characters from millions of Facebook images, however, fully convolutional CNN is employed for generating the lexicon free transcription of each word [6].”
From Object detection using YOLO: challenges, architectural successors, datasets and applications (2023) · cites this paper as [6]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 3×
“chitectural decision and a natural choice was to use a character-based recognition model. In the initial stages of development, we tested various efficient backbone architectures, including ResNet-18 [17] and SqueezeNet [18]. We evaluated these choices on our synthetic data and ended up selecting ResNet-18 as our main…”
MethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 3×
“Our text detection approach, based on Faster-RCNN model [24], is responsible of detecting regions of the image that contain text.”
MethodInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“ns. As we show empirically in Section 5, ShuffleNet is significantly faster than ResNet and did not lead to inferior accuracy. The ShuffleNet convolutional trunk is pre-trained using ImageNet dataset [10]. To train the end-to-end detection system, the model was bootstrapped with an in-house synthetic dataset and then…”
MethodShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982 · cited 1×
“e encoding, RPN, and classifiers) are trained jointly in a supervised, end-to-end manner. Our text detection model uses Faster-RCNN, but replaces the ResNet convolutional body with a ShuffleNet-based [31] architecture for efficiency reasons. As we show empirically in Section 5, ShuffleNet is significantly faster than…”
MethodShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201 · cited 2×
“st of the k fully connected layers that perform character classification in the CHAR model. Orthogonal to this, one can use different body architectures. We experimented with ResNet-18 and SqueezeNet [18], and obtained higher accuracy with ResNet-18 while experiencing only a very small overhead in computation. Image P…”
MethodGraves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“The second approach was based on curriculum learning [6], i.e ., starting with a simpler problem and increasing the difficulty as the model improves.”
BackgroundShi, B., Bai, X., & Yao, C. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2646371 · Cited by 3,172
Edge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656 · cited 1×
“Before the recognition stage, a combination of Edge Boxes [33] and an aggregate channel features (ACF) detector framework [11] is used to detect candidate locations for the words.”
Method
Research works printed but not linked: 1
- [2] Caffe2, 2017. -
Cited by
Items 201–300 of 425
A Study on Approaches for Automatic Number Plate Recognition (ANPR) Systems
Jadhav, A., Dongre, O. K., Shinde, T. K., Patil, D. S., & Dewan, J. H. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-16-5689-7_57 · Cited by 7
Exploring coherence from heterogeneous representations for OCR image captioning
Zhang, Y., Song, Z., & Hu, Z. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01470-1 · Cited by 6
“We use Rosetta [52] as our OCR system.”
Byun, J., Kang, B., Mun, D., Lee, G., & Kim, H. (2025) · Journal of Computational Design and Engineering · doi:10.1093/jcde/qwaf053 · Cited by 6
Open set text recognition technology
Yang, C. H., Liu, C., Fang, Z., Han, Z., Liu, C., & Yin, X.-C. (2023) · Journal of Image and Graphics · doi:10.11834/jig.230018 · Cited by 6
SUST and RUST: Two Datasets for Uyghur Scene Text Recognition
Kong, F., Liu, Y., Xu, M., Silamu, W., & Li, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3331213 · Cited by 6
“Deep learning divides the STR process into two subtasks: text detection [7], [8] and text recognition [9], [10], where the former is responsible for detecting text regions and the latter for performing character recognition.”
BackgroundMethodViTSTR-Transducer: Cross-Attention-Free Vision Transformer Transducer for Scene Text Recognition
Buoy, R., Iwamura, M., Srun, S., & Kise, K. (2023) · Journal of Imaging · doi:10.3390/jimaging9120276 · Cited by 6
Multiple-Question Multiple-Answer Text-VQA
Tang, P., Appalaraju, S., Manmatha, R., Xie, Y., & Mahadevan, V. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-industry.7 · Cited by 6
Ren, J., Liu, W., Chen, J., Zhu, X., Li, R., Zhao, T., Zhang, J., Yuan, T., Yin, S., Zhai, X., et al. (2025) · Remote Sensing · doi:10.3390/rs17020204 · Cited by 6
“…included CRNN (convolutional recurrent neural network, TPAMI 2016 ) [23], STAR‑Net (Spatial Attention Residue Network, BMVC 2016 ) [39], Rosetta [40], SAR (Show, Attend and Read, AAAI 2019 ) [41], SRN [42], SVTR (Single Visual Model for Scene Text Recognition, IJCAI 2022 ) [43], and the…”
Automatic Parsing and Extraction of Structured Data from Fax Images
Rizan, R. U. B. (2024) · International Conference on Engineering and Emerging Technologies (ICEET) · doi:10.1109/iceet65156.2024.10913905 · Cited by 6
“In the first approach, text detection and recognition are processed separately [4], [5], [6], [7], [8].”
A New Approach for Integrated Recognition and Correction of Texts from Images
Wei, J., Chen, K., He, J., Huang, Z., Lian, Y., & Zhou, Y. (2019) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2019.00104 · Cited by 6
“The state-of-the-art OCR system [4] also applied CTC in the decoder to generate final output label.”
MethodSNRNet: A Deep Learning-Based Network for Banknote Serial Number Recognition
Lin, Z., He, Z., Wang, P., Tan, B., Lu, J., & Bai, Y. (2020) · Neural Processing Letters · doi:10.1007/s11063-020-10313-9 · Cited by 6
End-to-End Analysis for Text Detection and Recognition in Natural Scene Images
Alnefaie, A., Gupta, D., Bhuyan, M. H., Razzak, I., Gupta, P. K., & Prasad, M. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207001 · Cited by 7
“Borisyuk et al. [17] develop an optical character recognition system to process images at Facebook scale.”
BackgroundAn End-to-End Marking Recognition System for PCB Optical Inspection
Ghosh, S., Bhavsar, J. S., Elsayed, N., & Asadizanjani, N. (2022) · IEEE Physical Assurance and Inspection of Electronics (PAINE) · doi:10.1109/paine56030.2022.10014764 · Cited by 6
“Numerous sequence-based text recognition systems (e.g., CRNN [25], ROSETTA [26], GRCNN [27], DTRN [28], etc.) have employed connectionist temporal classification (CTC) [29] for character sequence prediction, inspired by the speech recognition challenge.”
BackgroundYan, F., Silamu, W., Chai, Y., & Li, Y. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15418-6 · Cited by 6
“We use the Rosetta [6] OCR system to extract text marks on each image, identify up to 50 OCR tokens in the image, and then use FastText [5] to represent OCR tokens.”
Yan, F., Li, Z., Silamu, W., & Li, Y. (2023) · Machine Learning · doi:10.1007/s10994-023-06426-6 · Cited by 5
“Then, we utilize the Rosetta model (Borisyuk et al., 2018) to extract scene text from the images.”
Agrawal, M., Jalal, A. S., & Sharma, H. (2024) · Computational Intelligence · doi:10.1111/coin.12635 · Cited by 5
Fang, C., Jiang, W., Fang, Y., Peng, Y., & Liu, Y. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3577188 · Cited by 5
“2) OCR Features: For scene texts recognized in input images by the OCR system [33], we have three di ff erent features: 1) visual features extracted by Faster R-CNN [34] x ocr , fr i .”
Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models
Yoon, J. W., Kim, H. Y., Lee, H., Ahn, S., & Kim, N. S. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3297955 · Cited by 5
Li, W., Zhang, Y., Huang, Y., Shen, Y., & Wang, Z. (2023) · Applied Sciences · doi:10.3390/app13179539 · Cited by 5
Event and Activity Recognition in Video Surveillance for Cyber-Physical Systems
Bhaumik, S., Jana, P. K., & Mohanta, P. P. (2021) · Advances in Science, Technology & Innovation/Advances in science, technology & innovation · doi:10.1007/978-3-030-66222-6_4 · Cited by 5
“In such situations, text appearing on the video frame and/or subtitles can be extracted and recognized very efficiently by applying some classical image processing and classification techniques (Jana et al. 2017; Luo et al. 2019; Borisyuk et al. 2018).”
MethodMEAN: Multi - Element Attention Network for Scene Text Recognition
Yan, R., Peng, L., Xiao, S., Yao, G., & Min, J. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413166 · Cited by 5
An attention-based network for serial number recognition on banknotes
Lin, Z., He, Z., Tan, B., Shen, Y., Wang, P., & Liu, T. (2022) · Signal Processing Image Communication · doi:10.1016/j.image.2022.116732 · Cited by 5
Text Font Correction and Alignment Method for Scene Text Recognition
Ding, L., Liu, Y., Zhao, Q., & Liu, Y. (2024) · Sensors · doi:10.3390/s24247917 · Cited by 5
Image processing application in character recognition
Nanda, P. K., & Goswami, L. (2021) · Materials Today Proceedings · doi:10.1016/j.matpr.2021.03.697 · Cited by 5
Jain, P., Taneja, K., & Taneja, H. (2021) · Recent Patents on Engineering · doi:10.2174/1872212115666210715163919 · Cited by 5
Improving Long Handwritten Text Line Recognition with Convolutional Multi-way Associative MemoryPreprint
Nguyen, D. D., Tran, N., & Lê, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.01577 · Cited by 5
“Since first introduced, CRNN for Handwritten Text Recognition (HTR) has been constantly breaking state-of-the-art results [8, 31, 39], and being deployed in industrial applications [5, 6].”
BackgroundIntegrating vision and language: a novel approach for translation of low-resource Indic languages
Chauhan, S., Saxena, S., & Jain, S. (2025) · Asia Pacific Translation and Intercultural Studies · doi:10.1080/23306343.2025.2585418 · Cited by 4
MSTAR: Box-free Multi-query Scene Text Retrieval with Attention Recycling
Yin, L., Xie, X., Li, Z., Bai, X., & Liu, Y. (2025) · neural information processing systems · doi:10.52202/085713-1939 · Cited by 4
Ladder-side mixture of experts adapters for bronze inscription recognition
Zhou, R., Qiu, P., Zhang, Q., Li, C., & Yang, X. (2026) · npj Heritage Science · doi:10.1038/s40494-025-02294-6 · Cited by 4
From pixels to text: A deep learning survey of scene text detection and recognition
Kadha, V., Duddeti, B. B., Srinadh, K., Kumar, B. S., Janjanam, L., & Medhi, K. (2026) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2026.111139 · Cited by 4
Heng, H., Li, S., Li, P., Lin, Q., Chen, Y., & Zhang, L. (2023) · PLoS ONE · doi:10.1371/journal.pone.0294943 · Cited by 4
“Prior research [1–4] has introduced enhanced models for recognizing curved text images, employing a rectification module to rectify text images and recognition capabilities.”
BackgroundCorrelation-guided decoding strategy for low-resource Uyghur scene text recognition
Xu, M., Zhang, J., Xu, L., Silamu, W., & Li, Y. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01689-5 · Cited by 4
Language-Aware Non-autoregressive Khmer Textline Recognition
Buoy, R., Iwamura, M., Srun, S., & Kise, K. (2025) · Lecture notes in computer science · doi:10.1007/978-981-97-8702-9_23 · Cited by 4
The image and ground truth dataset of Mongolian movable-type newspapers for text recognition
Lu, M., Bao, F., Zhang, H., & Gao, G. (2023) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-023-00450-x · Cited by 4
RRecT: Chinese Text Recognition with Radical-Enhanced Recognition Transformer
Deng, X., Huang, Z., Ma, K., Chen, K., Guo, J., & Qiu, W. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44223-0_41 · Cited by 4
So Many Heads, So Many Wits: Multimodal Graph Reasoning for Text-Based Visual Question Answering
Zheng, W., Yan, L., & Wang, F. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3319964 · Cited by 4
“3) Text Feature: For texts recognized in the input images, we follow the M4C work and employ the Rosetta OCR system [61] to receive rich representations: where t is the feature with the pre-trained FastText [62] from the N T words’ tokens in the images; t is the pyramidal histogram of characters…”
MethodOscar: Omni-scale robust contrastive learning for Text-VQA
Yue, J., Bi, X., & Chen, Z. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124785 · Cited by 4
Dual Feature Enhanced Scene Text Recognition Method for Low-Resource Uyghur
Xu, M., Zhang, J., Xu, L., Li, Y., & Silamu, W. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8511-7_5 · Cited by 4
Generating Spatial-aware Captions for TextCaps
Wang, Y., & Shang, L. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956709 · Cited by 4
CONVJSSP: Convolutional Learning for Job-Shop Scheduling Problems
Wang, T., Payberah, A. H., & Vlassov, V. (2020) · International Conference on Machine Learning and Applications · doi:10.1109/icmla51294.2020.00229 · Cited by 4
Weakly Supervised Attention Rectification for Scene Text Recognition
Gu, C., Wang, S., Zhu, Y., Huang, Z., & Chen, K. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412037 · Cited by 5
“Many researchers have proposed various methods for scene text recognition [1-25].”
BackgroundEASTER: Simplifying Text Recognition using only 1D Convolutions
Chaudhary, K., & Bali, R. (2021) · doi:10.21428/594757db.65eda33c · Cited by 4
Devi, M. M., Maddala, S., & Raju, S. V. (2021) · Journal of Computing in Higher Education · doi:10.1007/s12528-021-09294-y · Cited by 4
A Deep Learning-Based Framework for End-To-End Identification of IMO Numbers on Ships
Ma, H., Zhang, D., Fan, L., Li, Y., & Xu, Z. (2023) · International Conference on Transportation Information and Safety · doi:10.1109/ictis60134.2023.10243842 · Cited by 4
Lifelong Scene Text Recognizer via Expert Modules
Xia, S., Geng, L., Liu, N., Sun, H., & Qin, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612062 · Cited by 4
“Many non-autoregressive models [6, 18, 28, 42] use the CTC loss, which is introduced to align the frame-wise probability with labels.”
MethodIntegration of YOLOv5 Algorithm and OpenCV in Innovative Smart Parking Management Approach
Hidayah, A. H., Zuhriyah, S., Asrul, B. E. W., Yuyun, Y., & Prakasa, E. (2024) · Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · doi:10.29207/resti.v8i3.5728 · Cited by 4
“Object recognition for these images is widely used to recognize various elements, such as texts [2], vehicles [3], fingerprints [4], faces [5], and others.”
BackgroundGraph Relation Transformer: Incorporating pairwise object features into the Transformer architecturePreprint
Yang, M., Anantharaman, A., Kitowski, Z., & Robert, D. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06075 · Cited by 4
“Namely, SMA uses a custom Rosetta-en OCR module and SAMT uses 2 extra Transformer layers, a Google OCR module and a ResNext R-CNN.”
MethodTraining Protocol Matters: Towards Accurate Scene Text Recognition via Training Protocol SearchingPreprint
Chu, X., Wang, Y., Shen, C., Chen, J., & Chu, W. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.06696 · Cited by 4
Mulaw, Y. (2024) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · Cited by 4
Towards reduced-complexity scene text recognition (RCSTR) through a novel salient feature selection
Buoy, R., Iwamura, M., Srun, S., & Kise, K. (2024) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-024-00474-x · Cited by 3
A Lightweight Detection and Recognition Framework for cigarette laser code
Zhou, W., Li, H., Zheng, L., Xiao, S., & Yi, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111777 · Cited by 3
A duet of perception and reasoning: CLIP and LLM brainstorming for scene text recognition
Jia, Z., Wang, J., Song, K., Wang, Z., Ma, X., & Jin, R. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.132236 · Cited by 3
Patel, G., Kim, T., Lin, Q., Allebach, J. P., & Qiu, Q. (2024) · Electronic Imaging · doi:10.2352/ei.2024.36.8.image-241 · Cited by 3
Tian, F., Wang, H., Wan, Z., Liu, R., Liu, R., Lv, D., & Lin, Y. (2024) · Electronics · doi:10.3390/electronics13101941 · Cited by 3
Parstr: partially autoregressive scene text recognition
Buoy, R., Iwamura, M., Srun, S., & Kise, K. (2024) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-024-00470-1 · Cited by 3
“Many methods based on a connectionist temporal classification (CTC) decoder [7–10] transcribe a visual-temporal feature sequence into a target character sequence in parallel.”
Scene-text aware cross-modal retrieval based on semantic matching (ChinaMM2024)
Cheng, S., Zhang, F., Zhang, X., & Sun, Z. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01481-y · Cited by 3
Hybrid Encoding Method for Scene Text Recognition in Low-Resource Uyghur
Xu, M., Zhang, J., Xu, L., Li, Y., & Silamu, W. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8511-7_7 · Cited by 3
Two-step sequence transformer based method for Cham to Latin script transliteration
Nguyen, T.-N., Burie, J.-C., Le, T., & Schweyer, A.-V. (2023) · International Workshop on Historical Document Imaging and Processing (HIP) · doi:10.1145/3604951.3605525 · Cited by 3
“We found that models such as [3] and [5], which only use CNNs or fully connected layer are not adapted to our data.”
MethodNhận dạng văn bản tiếng Việt trong ảnh ngoại cảnh bằng học sâu
Huy, H. V., Tân, N. T. T., & Tạo, N. Q. (2023) · Journal of Military Science and Technology · doi:10.54939/1859-1043.j.mst.90.2023.140-149 · Cited by 3
CLCRNet: An Optical Character Recognition Network for Cigarette Laser Code
Zhou, W., Zheng, L., Li, X., Yang, Z., & Yi, J. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3375403 · Cited by 3
Enhanced Object Detection Based on YOLOv9 for Marine Life
Pawar, A., Handa, A., Pujari, M., & Hema, D. D. (2024) · International Conference on Computational Intelligence and Communication Networks (CICN) · doi:10.1109/cicn63059.2024.10847536 · Cited by 3
“We referred to the advancements presented in paper published by M. Pawan Kumar et al. to address the efficiency issues identified in RCNN. [5, 6] Fast R-CNN introduced a more efficient pipeline by utilizing a fixed-size feature map extraction approach with a Region of Interest (ROI) pooling layer…”
Apvit: ViT with adaptive patches for scene text recognition
Zhang, N., Li, C., Wang, Z., Ma, J., & Feng, Z. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-06570-9 · Cited by 3
Mileage Extraction From Odometer Pictures for Automating Auto Insurance Processes
Acharya, S., & Fung, G. (2019) · Frontiers in Applied Mathematics and Statistics · doi:10.3389/fams.2019.00061 · Cited by 3
Unattached irregular scene text rectification with refined objective
Gong, Y., Deng, L., Zhang, Z., Duan, G., Ma, Z., & Xie, M. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.047 · Cited by 3
DM2S2: Deep Multimodal Sequence Sets With Hierarchical Modality Attention
Kitada, S., Iwazaki, Y., Togashi, R., & Iyatomi, H. (2022) · IEEE Access · doi:10.1109/access.2022.3221812 · Cited by 3
Scene Text Recognition for Text-Based Traffic Signs
Taki, Y., & Zemmouri, E. M. (2023) · Advances in transdisciplinary engineering · doi:10.3233/atde230010 · Cited by 3
Tesseract OCR Recognition Based on Arabic Machine-Printed Document
Ramteke, R. J., & Maamari, M. R. A. O. A. (2023) · Advances in intelligent systems research/Advances in Intelligent Systems Research · doi:10.2991/978-94-6463-196-8_27 · Cited by 3
N-FTRN: Neighborhoods based fully convolutional network for Chinese text line recognition
Li, H., Wang, W., & Lv, K. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-7410-1 · Cited by 3
“The CTC decoder [3, 6, 10, 25, 34, 35, 39] uses an extra “blank” label to indicate non-character and makes prediction for each frame of the encoded sequence, then translates the label sequence into a character sequence.”
MethodReADS: A Rectified Attentional Double Supervised Network for Scene Text Recognition
Song, Q., Jiang, Q., Li, N., Zhang, R., & Wei, X. K. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413343 · Cited by 3
“In industrial applications, Facebook conducts an OCR system called Rosetta [14] to process daily uploaded images.”
MethodFully Shareable Scene Text Recognition Modeling for Horizontal and Vertical Writing
Orihashi, S., Yamazaki, Y., Uchida, M., Takashima, A., & Masumura, R. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897836 · Cited by 3
“The continuous representations are then converted into a character string by using connectionist temporal classification (CTC) [5–9] or attention-based autoregressive decoder [10–12].”
MethodDeep Multi-Modal SetsPreprint
Reiter, A., Jia, M., Yang, P., & Lim, S.-N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01607 · Cited by 3
Assessing Individual and Community Vulnerability to Fake News in Social NetworksPreprint
Rath, B., Gao, W. G., & Srivastava, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.02434 · Cited by 3
“Use of convolution neural networks [3] and recurrent neural networks [17] and numerous variants of these fundamental models have shown promising results.”
CSTR: A Classification Perspective on Scene Text Recognition.Preprint
Cai, H., Sun, J., & Xiong, Y. (2021) · arXiv (Cornell University) · Cited by 3
Kobchaisawat, T. (2019) · Chula Digiverse · doi:10.58837/chula.the.2019.160 · Cited by 3
Multiple-Question Multiple-Answer Text-VQAPreprint
Tang, P., Appalaraju, S., Manmatha, R., Xie, Y., & Mahadevan, V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.08622 · Cited by 3
“Following (Biten et al., 2022; Powalski et al., 2021), we use Amazon Textract 2 , Amazon Text-in-Image 3 , and Rosetta (Borisyuk et al., 2018) to extract OCR results for document images ( i.e ., IDL and DocVQA images), non-document images (except for OCR-VQA images), and OCR-VQA images, respectively.”
MethodKim, J., Mun, D., Hwang, J., & Hwang, S. (2025) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-025-00525-x · Cited by 2
GAN Augmented Hybrid Transformer Network (GHTNet) For Ancient Tamil Stone Inscription Recognition
Murugan, B., & Visalakshi, P. (2025) · npj Heritage Science · doi:10.1038/s40494-025-02097-9 · Cited by 2
Improving Pseudo-Labeling by Dynamic Confidence Calibration for Semi-Supervised Sequence Recognition
Xu, K., Peng, Z., Huang, S., Zhang, G., Hu, Y., & Peng, W. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3789203 · Cited by 2
Domain-Complementary Prior With Fine-Grained Feedback for Scene Text Image Super-Resolution
Zhang, S., Li, Y., Dai, P., ZHOU, X., & Xie, G. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3657246 · Cited by 2
“Rosetta [45] employs a fully convolutional model optimized using the CTC loss.”
Ren, J., Liu, W., Chen, J., Li, Z., Zhang, J., & Yin, S. (2026) · Geo-spatial Information Science · doi:10.1080/10095020.2026.2630486 · Cited by 2
Video Big Data Analytics in the Cloud: Research Issues and Challenges
Alam, A., Khalid, S., Khan, M. N., Afridi, T. H., Ullah, I., & Lee, Y.-K. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-66840-2_108 · Cited by 2
Scalable Document Image Information Extraction with Application to Domain-Specific Analysis
Zheng, Y., Kong, S., Zhu, W., & Ye, H. (2019) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata47090.2019.9006064 · Cited by 2
“Rosetta [1] present by Facebook adapt Faster-RCNN [2] based text detection approach.”
MethodImproving Small License Plate Detection with Bidirectional Vehicle-Plate Relation
Dai, S., Chen, S.-L., Liu, Q., Zhu, C., Liu, Y.-J., Chen, F., & Yin, X.-C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53308-2_19 · Cited by 2
Knowledge-representation-enhanced multimodal Transformer for scene text visual question answering
Zhou, Y., Yu, J., Junjie, Z., & Kuang, Z. (2022) · Journal of Image and Graphics · doi:10.11834/jig.211213 · Cited by 2
Tan, Y., Chen, Y., & Wang, J. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3486104 · Cited by 2
Improvement in Efficiency of The State-Of-The-Art Handwritten Text Recognition Models
Et. al. Sai Suryateja, S. (2021) · Türk bilgisayar ve matematik eğitimi dergisi · doi:10.17762/turcomat.v12i10.5661 · Cited by 2
“In the past few years, the research in HTR has illustrated rigorous advancements over HMM. Deep learning methods such as Convolutional Recurrent Neural Networks (CRNN) have repeatedly improved and gave practical results in the industrial application [6].”
BackgroundOCR-based Inventory Management Algorithms Robust to Damaged Images
Seo, M., Kim, D., Kang, H., Cho, D., & Choi, D. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561280 · Cited by 2
“Additionally, performance analysis of text containing real-world damaged images in real-world applications was overlooked, such as in [7], [8], and [9].”
BackgroundInfluentialTowards Multilingual Image Captioning Models that Can Read
García, R., Beltrán, B., Hernández, C., & Ayala, D. V. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89820-5_2 · Cited by 2
Searching for memory-lighter architectures for OCR-augmented image captioning
Gallardo-García, R., Beltrán, B., Hernández, C., & Ayala, D. V. (2021) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-219230 · Cited by 2
“The reading module of the M4C-Captioner is based on the Rosetta [4] large scale system for text detection and recognition.”
InfluentialLocalization of Pointed-At Word in Printed Documents via a Single Neural Network
Rubin, Z. H. A. O., ZHENG, X., Ying, Z., & Fan, L. (2022) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2021edp7143 · Cited by 2
Intelligent Scene Text Recognition in Streaming Videos
Ha, M.-Q., Phan, V.-H., Nguyen, B. D., Nguyen, H.-B., Do, T.-H., Pham, Q.-D., & Dao, N. (2022) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-15063-0_34 · Cited by 2
Text-Level Contrastive Learning for Scene Text Recognition
Zhuang, J., Ren, Y., Li, X., & Liang, Z. (2022) · International Conference on Asian Language Processing · doi:10.1109/ialp57159.2022.9961322 · Cited by 2
Re-Thinking Text Clustering for Images with Text
Mishra, S. K., Joshi, S., & Gopalakrishnan, V. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41679-8_16 · Cited by 2
Reading Chinese in Natural Scenes with a Bag-of-Radicals PriorPreprint
Liu, Y., Qingjie, L., Chen, J., & Yunhong, W. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.02576 · Cited by 2
Scene Text Recognition with SemanticsPreprint
Placidi, J. C., Miao, Y., Wang, Z., & Specia, L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.10836 · Cited by 2
“The CRNN [30], Rosetta [3], STAR-Net [19] and TPS-ResNetBiLSTM-Attn [1] are the benchmark end-to-end neural models for scene text recognition.”
BackgroundMethodUsing Visual Cropping to Enhance Fine-Detail Question Answering of BLIP-Family ModelsPreprint
Zhang, J., Khayatkhoei, M., Chhikara, P., & Ilievski, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.00228 · Cited by 2
“Namely, we take the Optical Character Recognition (OCR) (Borisyuk et al., 2018) annotations from the dataset and keep the questions where the OCR result is perfectly aligned with ground truth in a single bounding box.”
MethodHiremath, A., Irabatti, N., Desai, A., Dhondge, P., & Sheikh, S. (2024) · Research Square · doi:10.21203/rs.3.rs-4301899/v1 · Cited by 2
Machine Learning for Measuring and Analyzing Online Social Communications
Bronk, C., Lendasse, A., Lindner, P., Wallach, D. S., & Hammer, B. (2021) · European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN) · doi:10.14428/esann/2021.es2021-3 · Cited by 1
S5TR: Simple Single Stage Sequencer for Scene Text Recognition
Wu, Z., Li, J., & Xu, J. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8391-9_11 · Cited by 1
Using CRAFT algorithm-based OCR system to improve multilanguage text detection and recognition
Fadhil, A. H., & Altaei, M. S. M. (2024) · AIP conference proceedings · doi:10.1063/5.0236338 · Cited by 1
Metadata
- Authors
- Borisyuk, F., Gordo, A., & Sivakumar, V.
- Affiliations
- Meta (United States) (US)
- Venue
- Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2018-07-19
- Pages
- 71-79
- OpenAlex
- W2809273748
- Cited by
- 353
- References
- 28
- Citations by year
- 2026: 13 · 2025: 35 · 2024: 44 · 2023: 65 · 2022: 51 · 2021: 79 · 2020: 38 · 2019: 26 · 2018: 2
- FWCI
- 10.92
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Vehicle License Plate Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- optical character recognition, Rosetta, Facebook, system architecture, text extraction from images, media understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.