Rosetta: Large Scale System for Text Detection and Recognition in Images
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
In this paper we present a deployed, scalable optical character recognition (OCR) system, which we call Rosetta , designed to process images uploaded daily at Facebook scale. Sharing of image content has become one of the primary ways to communicate information among internet users within social networks such as Facebook, and the understanding of such media, including its textual information, is of paramount importance to facilitate search and recommendation applications. We present modeling techniques for efficient detection and recognition of text in images and describe Rosetta 's system architecture. We perform extensive evaluation of presented technologies, explain useful practical approaches to build an OCR system at scale, and provide insightful intuitions as to why and how certain components work based on the lessons learnt during the development and deployment of the system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A novel OCR system is developed, named as Rosetta, wherein faster-RCNN is utilized for detecting the text characters from millions of Facebook images, however, fully convolutional CNN is employed for generating the lexicon free transcription of each word [6].”
From Object detection using YOLO: challenges, architectural successors, datasets and applications (2023) · cites this paper as [6]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 3×
“chitectural decision and a natural choice was to use a character-based recognition model. In the initial stages of development, we tested various efficient backbone architectures, including ResNet-18 [17] and SqueezeNet [18]. We evaluated these choices on our synthetic data and ended up selecting ResNet-18 as our main…”
MethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 3×
“Our text detection approach, based on Faster-RCNN model [24], is responsible of detecting regions of the image that contain text.”
MethodInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“ns. As we show empirically in Section 5, ShuffleNet is significantly faster than ResNet and did not lead to inferior accuracy. The ShuffleNet convolutional trunk is pre-trained using ImageNet dataset [10]. To train the end-to-end detection system, the model was bootstrapped with an in-house synthetic dataset and then…”
MethodShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982 · cited 1×
“e encoding, RPN, and classifiers) are trained jointly in a supervised, end-to-end manner. Our text detection model uses Faster-RCNN, but replaces the ResNet convolutional body with a ShuffleNet-based [31] architecture for efficiency reasons. As we show empirically in Section 5, ShuffleNet is significantly faster than…”
MethodShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201 · cited 2×
“st of the k fully connected layers that perform character classification in the CHAR model. Orthogonal to this, one can use different body architectures. We experimented with ResNet-18 and SqueezeNet [18], and obtained higher accuracy with ResNet-18 while experiencing only a very small overhead in computation. Image P…”
MethodGraves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“The second approach was based on curriculum learning [6], i.e ., starting with a simpler problem and increasing the difficulty as the model improves.”
BackgroundShi, B., Bai, X., & Yao, C. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2646371 · Cited by 3,172
Edge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656 · cited 1×
“Before the recognition stage, a combination of Edge Boxes [33] and an aggregate channel features (ACF) detector framework [11] is used to detect candidate locations for the words.”
Method
Research works printed but not linked: 1
- [2] Caffe2, 2017. -
Cited by
Items 101–200 of 425
VCR: A Task for Pixel-Level Complex Reasoning in Vision Language Models via Restoring Occluded Text
Zhang, T., Wang, S., Li, L., Zhang, G., Taslakian, P., Rajeswar, S., Fu, J., Liu, B., & Bengio, Y. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2406.06462 · Cited by 24
“Besides, understanding these neural mechanisms can inspire new algorithms capable of mimicking human-like perception and interpretation in dynamic, real-world conditions where occlusion is common.”
PiF: in-flash acceleration for data-intensive applications
Chun, M., Lee, J., Lee, S., Kim, M., & Kim, J. (2022) · ACM Workshop on Hot Topics in Storage and File Systems (HotStorage) · doi:10.1145/3538643.3539742 · Cited by 23
“, an intelligent query app [1, 2]) should handle a very large amount of data (e.”
BackgroundEnhancing handwritten text recognition accuracy with gated mechanisms
Chinthaginjala, R., Dhanamjayulu, C., Kim, T.-H., Ahmed, S., Kim, S.-Y., Kumar, A., Annepu, V., & Ahmad, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-67738-8 · Cited by 23
TRIE++: Towards End-to-End Information Extraction from Visually Rich DocumentsPreprint
Cheng, Z., Zhang, P., Li, C., Liang, Q., Xu, Y., Li, P., Pu, S., Niu, Y., & Wu, F. Q. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.06744 · Cited by 23
The Best Protection is Attack: Fooling Scene Text Recognition With Minimal Pixels
Xu, Y., Dai, P., Li, Z., Wang, H., & Cao, X. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3245984 · Cited by 22
“Compared with the attack method in [29] when attacking Rosetta [32], our method also achieves 100% SR on IC13, IC15, and CUTE datasets.”
MethodInfluentialUTRNet: High-Resolution Urdu Text Recognition in Printed Documents
Rahman, A., Ghosh, A., & Arora, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41734-4_19 · Cited by 22
Sharma, H., & Jalal, A. S. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116159 · Cited by 22
Afkari-Fahandari, A., Shabaninia, E., Asadi-Zeydabadi, F., & Nezamabadi–pour, H. (2025) · ACM Computing Surveys · doi:10.1145/3771273 · Cited by 22
“At every time step of the OCR process, the CTC layer accepts the per-frame predictions made by the recurrent layers as input and generates a probability distribution over all conceivable label sequences that may have generated the input sequence.”
Leveraging Model Fusion for Improved License Plate Recognition
Laroca, R., Junior, L. A. Z., Estevam, V., Minetto, R., & Menotti, D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-49249-5_5 · Cited by 21
“We explore 12 recognition models in our experiments: RARE [35], R 2 AM [25], STAR-Net [28], CRNN [34], GRCNN [42], Holistic-CNN [39], Multi-task-LR [11], Rosetta [4], TRBA [2], CR-NET [37], Fast-OCR [23] and ViTSTR-Base [1].”
MethodAn Empirical Study of Scaling Law for OCRPreprint
Rang, M., Bi, Z., Liu, C., Wang, Y., & Han, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.00028 · Cited by 21
“In terms of OCR, we utilized Rosetta [10] for detection, and CLIP4STR-L* for recognition.”
MethodDeep Learning-Powered System for Real-Time Digital Meter Reading on Edge Devices
Carvalho, R., Melo, J., Graça, R., Santos, G., & Vasconcelos, M. J. M. (2023) · Applied Sciences · doi:10.3390/app13042315 · Cited by 20
“Facebook’s Rosetta [15] is an efficient text recognition model, based on the ResNet18 [14] architecture.”
MethodA light-weight natural scene text detection and recognition system
Ghosh, J., Talukdar, A. K., & Sarma, K. K. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15696-0 · Cited by 20
“The data of parameters in Tables 6 and 7 of Rosetta [6] are obtained from [30].”
Digital Peter: New Dataset, Competition and Handwriting Recognition Methods
Potanin, M., Dimitrov, D., Shonenkov, A., Bataev, V., Karachev, D. K., Novopoltsev, M., & Chertok, A. V. (2021) · doi:10.1145/3476887.3476892 · Cited by 20
Recognition of expiry data on food packages based on improved DBNet
Zheng, J., Li, J., Ding, Z., Kong, L., & Chen, Q. (2023) · Connection Science · doi:10.1080/09540091.2023.2202363 · Cited by 20
“The CNN can acquire the features of each character in the image and simulate the interaction between characters very well (Borisyuk et al., 2018), and it is more computationally efficient.”
BackgroundMethodL umos : Empowering Multimodal LLMs with Scene Text Recognition
Shenoy, A., Lu, Y., Jayakumar, S., Chatterjee, D., Moslehpour, M., Chuang, P., Harpale, A., Bhardwaj, V., Xu, D., Zhao, S., et al. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671633 · Cited by 19
FVI: An End-to-end Vietnamese Identification Card Detection and Recognition in Images
Liem, H. D., Minh, N. Đ., Trung, N. B., Duc, H. T., Hiep, P. H., Dung, D. V., & Vu, D. H. (2018) · NAFOSTED Conference on Information and Computer Science (NICS) · doi:10.1109/nics.2018.8606831 · Cited by 19
“Different from the original paper, we adapted the cirriculum learning technique as training schedule from [5].”
BackgroundMethodTransformer text recognition with deep learning algorithm
Chen, Y., Shu, H., Xu, W., Yang, Z., Hong, Z., & Dong, M. (2021) · Computer Communications · doi:10.1016/j.comcom.2021.04.031 · Cited by 19
From Token to Word: OCR Token Evolution via Contrastive Learning and Semantic Matching for Text-VQA
Jin, Z., Shou, M. Z., Zhou, F., Tsutsui, S., Qin, J., & Yin, X.-C. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547977 · Cited by 19
“Text-based Visual Question Answering (Text-VQA) [6, 32] is a question-answering task that needs to read the text within the image [8, 15, 43], recognize the objects in the scene [30, 38–40], and answer the given question.”
BackgroundMethodDeep-learning generation of POI data with scene images
Zhang, J., Liu, X., Liao, W., & Li, X. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.04.004 · Cited by 18
Li, H., Wang, J., Du, X., Hu, Z., & Yang, S. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103106 · Cited by 18
Scene text understanding: recapitulating the past decade
Ghosh, M., Mukherjee, H., Obaidullah, S. M., Gao, X., & Roy, K. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10530-3 · Cited by 18
Improving Rotated Text Detection with Rotation Region Proposal NetworksPreprint
Huang, J., Sivakumar, V., Mnatsakanyan, M., & Pang, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07031 · Cited by 18
Self-citationRevisiting Classification Perspective on Scene Text RecognitionPreprint
Cai, H., Sun, J., & Xiong, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10884 · Cited by 18
Laroca, R., Estevam, V., Moreira, G. J. P., Minetto, R., & Menotti, D. (2025) · IET Intelligent Transport Systems · doi:10.1049/itr2.70086 · Cited by 17
“…(Liu et al., 2016) Scene Text Recognition CRNN (Shi et al., 2017) Scene Text Recognition GRCNN (Wang & Hu, 2017) Scene Text Recognition Rosetta (Borisyuk et al., 2018) Scene Text Recognition TRBA (Baek et al., 2019) Scene Text Recognition ViTSTR-Base (Atienza, 2021) Scene Text Recognition…”
Mohsenzadegan, K., Tavakkoli, V., & Kyamakya, K. (2022) · Sensors · doi:10.3390/s22166025 · Cited by 17
“Therefore, they involved recurrent neural layers (i.e., RNNs) [40] and used the so-called connectionist temporal classification (CTC) loss [41].”
MethodTowards Reasoning Ability in Scene Text Visual Question Answering
Wang, Q., Xiao, L., Lu, Y., Jin, Y., & He, H. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475390 · Cited by 17
“Here, as shown in Figure 1, “see" and “read" represent detecting and recognizing visual objects and scene text from images, which is usually handled by off-the-shelf object detectors and OCR engines like Faster RCNN [26] and Rosetta [5], etc.”
BackgroundMethodInfluentialTowards Increased Accessibility of Meme Images with the Help of Rich Face Emotion Captions
Prajwal, K. R., Jawahar, C. V., & Kumaraguru, P. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350939 · Cited by 17
“The emotion captions generated by our model complements existing works on meme text recognition [7] to yield accurate, meaningful descriptions for Reaction memes, thus making one of the most popular classes of memes readily accessible to the visually impaired users for the first time.”
BackgroundMethodInfluentialA Vertical Text Spotting Model for Trailer and Container Codes
Zhang, R., Bahrami, Z., & Liu, Z. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3115211 · Cited by 16
Filling in the Blank: Rationale-Augmented Prompt Tuning for TextVQA
Zeng, G., Zhang, Y., Zhou, Y., Fang, B., Zhao, G., Wei, X., & Wang, W. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612520 · Cited by 16
“Following previous work [3, 18], we employ the Rosetta-en [5] as the OCR module.”
MethodRobust Handwriting Recognition with Limited and Noisy Data
Pham, H., Setlur, A., Dingliwal, S., Lin, T.-H., Póczos, B., Huang, K., Li, Z., Lim, J., McCormack, C., & Vu, T. (2020) · International Conference on Frontiers in Handwriting Recognition · doi:10.1109/icfhr2020.2020.00062 · Cited by 16
“Finally, the most related approaches to our model are those designed for scene-text detection and recognition [1], [4], [8], [28], [39] although their problem is different from HWR.”
MethodQi, X., Chen, Y., Xiao, R., Li, C.-G., Zou, Q., & Cui, S. R. (2019) · International Conference on Document Analysis and Recognition Workshops (ICDARW) · doi:10.1109/icdarw.2019.40086 · Cited by 16
“Many companies have built their own OCR systems, such as Microsoft Azure OCR, PhotoOCR [2] and Attention-OCR [37] from Google, and Rosetta [3] from Facebook.”
BackgroundManaging Scalable Direct Storage Accesses for GPUs with GoFS
Li, S., Zhou, Y., Xue, Y., Xu, Y., & Huang, J. (2025) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3731569.3764857 · Cited by 15
Occluded Text Detection and Recognition in the Wild
Raisi, Z., & Zelek, J. (2022) · Canadian Conference on Computer and Robot Vision · doi:10.1109/crv55824.2022.00026 · Cited by 15
Zhuo, S., Zhang, X., Chen, Z., Wei, W., Wang, F., Li, Q., & Guan, Y. (2023) · Applied Sciences · doi:10.3390/app132011493 · Cited by 15
End-to-end video subtitle recognition via a deep Residual Neural Network
Yan, H., & Xu, X. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.01.019 · Cited by 15
DEED: Dynamic Early Exit on Decoder for Accelerating Encoder-Decoder Transformer Models
Tang, P., Zhu, P., Li, T., Appalaraju, S., Mahadevan, V., & Manmatha, R. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.9 · Cited by 15
SVIPTR: Fast and Efficient Scene Text Recognition with Vision Permutable Extractor
Cheng, X., Zhou, W., Li, X., Yang, J., Zhang, H., Sun, T., Zhang, W., Mai, Y., Li, T., Chen, X., et al. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679618 · Cited by 14
On Calibration of Scene-Text Recognition Models
Slossberg, R., Anschel, O., Markovitz, A., Litman, R., Aberdam, A., Tsiper, S., Mazor, S., Wu, J., & Manmatha, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25069-9_18 · Cited by 14
Almousawee, E., & Abbadi, N. K. E. (2022) · Al-Salam Journal for Engineering and Technology · doi:10.55145/ajest.2023.01.01.002 · Cited by 14
InstructOCR: Instruction Boosting Scene Text Spotting
Duan, C., Jiang, Q., Fu, P., Chen, J., Li, S., Wang, Z., Guo, S., & Luo, J. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i3.32286 · Cited by 13
MUST-VQA: MUltilingual Scene-Text VQA
Vivoli, E., Biten, A. F., Mafla, A., Karatzas, D., & Gómez, L. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25069-9_23 · Cited by 13
RRTrN: A lightweight and effective backbone for scene text recognition
Zhou, Q., Gao, J., Yuan, Y., & Wang, Q. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122769 · Cited by 13
M$^{3}$ANet: Multi-Modal and Multi-Attention Fusion Network for Ship License Plate Recognition
Zhou, C., Liu, D., Wang, T., Tian, J., & Cao, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3342694 · Cited by 13
“…despite the popularity of text-related recognition research in computer vision (CV) community, such as scene text recognition (STR) [1], [2], [3], [4], The authors are with the Machine Learning and I-health International Co-operation Base of Zhejiang Province, Hangzhou Dianzi University,…”
BackgroundDecoder Pre-Training with only Text for Scene Text Recognition
Zhao, S., Du, Y., Chen, Z., & Jiang, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681390 · Cited by 13
“Language-free methods predict characters directly from image features, with examples including CTC-based [22] methods like CRNN [51], SVTR [18] and Rosetta [6], ViT-based methods like ViTSTR [3], and methods that consider scene text recognition as an image classification problem [8, 29].”
Multi-Granularity Prediction with Learnable Fusion for Scene Text Recognition
Da, C., Wang, P., & Yao, C. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02653-7 · Cited by 13
Multi-modal Contextual Graph Neural Network for Text Visual Question Answering
Liang, Y., Wang, X. E., Duan, X., & Zhu, W. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412891 · Cited by 13
“For the scene texts, we use text tokens and bounding boxes extracted by Rosetta OCR system [5] provided in TextVQA dataset.”
MethodInfluentialRandom Blur Data Augmentation for Scene Text Recognition
Mu, D., Sun, W. M., Xu, G., & Li, W. (2021) · IEEE Access · doi:10.1109/access.2021.3117035 · Cited by 13
“Involved methods include CRNN [36], R2AM [16], GRCNN [50], Rosetta [49], RARE [14], STAR-Net [13] and the baseline [47] in this paper.”
MethodScene-Text Aware Image and Text Retrieval with Dual-Encoder
Miyawaki, S., Hasegawa, T., Nishida, K., Kato, T., & Suzuki, J. (2022) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/2022.acl-srw.34 · Cited by 13
Visual Question Answering: A Survey on Techniques and Common Trends in Recent LiteraturePreprint
dos Santos, C. F. G., Bastos, F. D. C., de Faria, A. C. A. M., da Silva, J. V. N. A., Uchôa, V. D. S., & Neto, D. G. D. A. (2023) · Research Square · doi:10.21203/rs.3.rs-3015858/v1 · Cited by 13
Explainable Connectionist-Temporal-Classification-Based Scene Text Recognition
Buoy, R., Iwamura, M., Srun, S., & Kise, K. (2023) · Journal of Imaging · doi:10.3390/jimaging9110248 · Cited by 12
SignboardText: Text Detection and Recognition in In-the-Wild Signboard Images
Do, T., Tran, T., Le, H., Nguyen, T., Ngo, T. D., & Duy-Dinh, L. (2021) · IEEE Access · doi:10.1109/access.2024.3395374 · Cited by 12
Source printer identification using printer specific pooling of letter descriptors
Joshi, S., Gupta, Y. K., & Khanna, N. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116344 · Cited by 12
Research on Energy Efficiency Management of Forklift Based on Improved YOLOv5 Algorithm
Li, Z., Lü, K., Zhang, Y., Li, Z., & Liu, J. (2021) · Journal of Mathematics · doi:10.1155/2021/5808221 · Cited by 12
“95, which are related to both precision and recall, as indexes to quantitatively judge whether the object detection methods meet accuracy and speed requirements [25].”
BackgroundOCR-oriented Master Object for Text Image Captioning
Tang, W., Hu, Z., Song, Z., & Hong, R. (2022) · International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531431 · Cited by 12
“Although the significant progress in the conventional image captioning [29] [3] [9] and OCR[14] [5] has between objects and OCR tokens to enrich the embedding.”
BackgroundEntity Graph Alignment and Visual Reasoning for Multimodal Fake News Detection
Li, G. S., Hu, D., Fu, X., Tang, Q., Wu, Y., Zhang, X., & Lyu, H. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754591 · Cited by 11
Borisyuk, F., Malreddy, S., Mei, J., Liu, Y., Liu, X., Maheshwari, P., Bell, A., & Rangadurai, K. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467081 · Cited by 11
“To handle such media we have developed an in-house optical character recognition system (OCR) called Rosetta [3].”
MethodSelf-citationNavigating Copyright for Libraries: Purpose and Scope
Coates, J., Owen, V., & Reilly, S. (2022) · De Gruyter eBooks · doi:10.1515/9783110732009 · Cited by 14
A Machine Vision-Based Character Recognition System for Suspension Insulator Iron Caps
Zhang, C., Liu, B., Chen, Z., Yan, J., Liu, F., Yu, W. W., & Zhang, Q. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3300474 · Cited by 11
“In the field of character recognition, researchers are also enthusiastic about using artificial intelligence and machine vision techniques [6], [7], [8], [9], [10], [11].”
BackgroundCross-region feature fusion with geometrical relationship for OCR-based image captioning
Zhou, J., Yang, C. Z., Zhu, Y., & Zhang, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128197 · Cited by 10
ProTegO: Protect Text Content against OCR Extraction Attack
He, Y., Chen, K., Chen, G., Ma, Z., Zhang, K., Zhang, J., Bian, H., Fang, H., Zhang, W., & Yu, N. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612076 · Cited by 10
“STAR-Net [20] (S-5a/5b) CRNN [29] (S-5a/5b) Rosetta [3] (S-5a/5b) RARE [30] (S-5a/5b) TRBA [1] Note that, “S-5a/5b” represents the underpainting Style 5 with white font and black font respectively, “base” means the naive generator trained without L 𝑔𝑢𝑖𝑑𝑒 and enhancement layer 𝐸 , and “*”…”
BackgroundMethodDEVICE: Depth and Visual Concepts Aware Transformer for OCR-based image captioning
Xu, D., Huang, Q., Zhang, X., Cheng, H., Shuang, F., & Cai, Y. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111522 · Cited by 10
“To ensure high quality OCR tokens, following LSTM-R [7], we apply external OCR systems [4, 31] to obtain both OCR tokens { a ocrm } m = 1: M and corresponding bounding box coordinates { b ocrm } m = 1: M , respectively.”
Wu, S., & Chang, F. (2025) · Alexandria Engineering Journal · doi:10.1016/j.aej.2025.05.037 · Cited by 10
3D Rendering Framework for Data Augmentation in Optical Character Recognition : (Invited Paper)
Spruck, A., Hawesch, M., Maier, A., Rieß, C., Seiler, J., & Kau, A. (2021) · International Symposium on Signals, Circuits and Systems · doi:10.1109/isscs52333.2021.9497438 · Cited by 10
“Rapid advances in machine learning and especially neural networks spur the development of increasingly robust OCR applications [1].”
BackgroundZero-TextCap: Zero-shot Framework for Text-based Image Captioning
Xu, D., Zhao, W., Cai, Y., & Huang, Q. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612571 · Cited by 10
Detecting Spam Images with Embedded Arabic Text in Twitter
Imam, N. H., & Vassilakis, V. G. (2019) · International Conference on Document Analysis and Recognition Workshops (ICDARW) · doi:10.1109/icdarw.2019.50107 · Cited by 10
“Recentlly, a study by [9] developed an OCR system to detect and recognize text in images uploaded to Facebook.”
BackgroundMethodInfluentialWeakly-supervised Fingerspelling Recognition in British Sign Language VideosPreprint
Prajwal, K. R., Bull, H., Momeni, L., Albanie, S., Varol, G., & Zisserman, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.08954 · Cited by 10
“…partly takes inspiration from the Transpotter (and more broadly from prior works for text spotting that detect words and learn to read them via CTC [4, 29]), wherein we process video features with a single Transformer encoder and then employ multiple heads to solve related tasks such as detection…”
ETDR: An Exploratory View of Text Detection and Recognition in Images and Videos
Chaitra, Y. L., Ramegowda, D., Gopalakrishna, M. T., Vijaya, A. P. B., & Shivananjappa, M. H. (2021) · Revue d intelligence artificielle · doi:10.18280/ria.350504 · Cited by 9
Data Augmentation via Adversarial Networks for Optical Character Recognition/Conference Submissions
Storchan, V., & Beauschene, J. (2019) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2019.00038 · Cited by 9
“[Borisyuk et al.(2018)] F. Borisyuk, A. Gordo, and V Sivakumar.”
MethodInfluentialObject Detection in Natural Scene Images Using Thresholding Techniques
Rani, N. S., Kavyashree, S., & Harshitha, R. (2020) · International Conference on Intelligent Computing and Control Systems (ICICCS) · doi:10.1109/iciccs48265.2020.9120992 · Cited by 9
“Borisyuk et al. [4] states the increase in the usage of images in the current generation and provides the Rosetta system where the process is divided into two different steps.”
BackgroundTwo-Stage Multimodality Fusion for High-Performance Text-Based Visual Question Answering
Li, B., Wang, J., Zhao, M., & Zhou, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_39 · Cited by 9
Revisiting multi-dimensional classification from a dimension-wise perspective
Shi, Y., Ye, H.-J., Man, D., Han, X., Zhan, D., & Jiang, Y. Y. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-023-3272-9 · Cited by 9
Automatic Virtual 3D City Generation for Synthetic Data Collection
Shen, B., Li, B., & Scheirer, W. J. (2021) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw52041.2021.00022 · Cited by 9
“We draw inspiration from this work [6] and trained a scene text detector with the dataset collected.”
MethodDavarOCR: A Toolbox for OCR and Multi-Modal Document Understanding
Liang, Q., Jiang, H., Chen, Y., Li, C., Li, P., Li, Z., Zou, B., Guo, D., Xu, Y., Xu, Y., et al. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548547 · Cited by 9
“Tesseract , chi-neseOCR 2 , chineseOCR_lite 3 , EasyOCR 4 are some earlier OCR PaddleOCR DB[18],EAST[37], etc.(4) CRNN[25],Rosetta[2] etc.(7) etc.(6) CRNN[25],ABINet[6] etc.(8) CRNN[25], SPIN[33], etc.(5) TP[22], MANGO[20], M-RCNN-e2e YORO [4] Chargrid[13] toolboxes.”
BackgroundA hybrid deep-based model for scene text detection and recognition in meter reading
Alshawi, A. A. A., Tanha, J., Balafar, M. A., & Imanzadeh, S. (2023) · International Journal of Information Technology · doi:10.1007/s41870-023-01383-8 · Cited by 9
“Rosetta, Facebook’s scalable OCR solution, was presented by Borisyuk et al. [22].”
MethodOn Calibration of Scene-Text Recognition ModelsPreprint
Slossberg, R., Anschel, O., Markovitz, A., Litman, R., Aberdam, A., Tsiper, S., Mazor, S., Wu, J., & Manmatha, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.12643 · Cited by 9
“[1] proposed a four-stage framework unifying several previous techniques [4, 42, 43, 28].”
What Large Language Models Bring to Text-rich VQA?Preprint
Liu, X., Tang, W., Ni, X., Lu, J., Zhao, R., Li, Z., & Tan, F. Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.07306 · Cited by 9
“In the visual module, we leverage OCR models (Du et al., 2021; Kim et al., 2022; Borisyuk et al., 2018b) to extract texts from images.”
BackgroundMethodA 307-fps 351.7-GOPs/W Deep Learning FPGA Accelerator for Real-Time Scene Text Recognition
Zhao, S., An, F., & Yu, H. (2019) · 2019 International Conference on Field-Programmable Technology (ICFPT) · doi:10.1109/icfpt47387.2019.00043 · Cited by 8
“Whereas, it has been recently improved by the development of DNNs, e.g., CRNN [1], STAR-Net [2], and Rosetta [3].”
ResultScene text visual question answering by using YOLO and STN
Nourali, K., & Dolkhani, E. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-023-10081-6 · Cited by 8
“Rosetta is Facebook’s real-time large-scale OCR system that can extract text from images and process a large volume of images uploaded to Facebook (Borisyuk et al., 2018).”
InfluentialAn End-to-End Scene Text Recognition for Bilingual Text
Albalawi, B. M., Jamal, A. T., Khuzayem, L. A., & Alsaedi, O. A. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8090117 · Cited by 8
Tanuwijaya, E., & Fatichah, C. (2020) · Briliant Jurnal Riset dan Konseptual · doi:10.28926/briliant.v5i1.434 · Cited by 8
Sharma, H., & Srivastava, S. (2021) · The Imaging Science Journal · doi:10.1080/13682199.2022.2153489 · Cited by 8
NRSTRNet: A Novel Network for Noise-Robust Scene Text Recognition
Yue, H., Huang, Y., Vong, C., Jin, Y., Zeng, Z., Yu, M., & Chen, C. (2023) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-023-00181-1 · Cited by 8
Memory-Efficient Models for Scene Text Recognition via Neural Architecture Search
Hong, S., Kim, D., & Choi, M.-K. (2020) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw50321.2020.9096928 · Cited by 8
Baruah, A., Das, K., Barbhuiya, F. A., & Dey, K. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.112 · Cited by 8
“Borisyuk et al. (2018) describes Facebook’s OCR system for extracting text from images in realtime.”
BackgroundSTRIDE: Scene Text Recognition In-Device
Munjal, R. S., Prabhu, A. D., Arora, N., Moharana, S., & Ramena, G. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534319 · Cited by 8
“Training CTC is hard and the models take a lot of time to converge [36].”
BackgroundLook, Read and Ask: Learning to Ask Questions by Reading Text in Images
Jahagirdar, S., Gangisetty, S., & Mishra, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86549-8_22 · Cited by 8
“3 For one of the datasets namely TextVQA, OCR-tokens extracted from Rosetta [6] are provided with the dataset.”
MethodInfluentialLess Is Better: Fooling Scene Text Recognition with Minimal Perturbations
Xu, Y., Dai, P., & Cao, X. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-92310-5_62 · Cited by 8
A survey on methods, datasets and implementations for scene text spotting
Blanco‐Medina, P., Fidalgo, E., Alegre, E., & González‐Castro, V. (2022) · IET Image Processing · doi:10.1049/ipr2.12574 · Cited by 8
Soares, A. G., Bezerra, B. L. D., & Lima, E. B. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86198-8_15 · Cited by 8
Digital Peter: Dataset, Competition and Handwriting Recognition MethodsPreprint
Potanin, M., Dimitrov, D., Shonenkov, A., Bataev, V., Karachev, D. K., & Novopoltsev, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.09354 · Cited by 8
Yıldız, S. (2024) · Engineering Science and Technology an International Journal · doi:10.1016/j.jestch.2024.101881 · Cited by 7
Fast and Lightweight Automatic Shipping Container Attributes Spotting
Cheng, R., You, Z., Chea, S., Wu, G., Xia, K., & Huang, S. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3573765 · Cited by 7
“Furthermore, when compared with models such as CRNN [32] and Rosetta [31], which achieve slightly higher FPS but lower accuracies (93.15% and 92.82%, respectively) and have higher computational requirements, SVTR-R stands out in both performance and e ffi ciency.”
InfluentialAccurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillation
Maracani, A., Ozkan, S., Cho, S., Kim, H., Noh, E., Min, J., Min, C. J., Park, D., & Ozay, M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01353 · Cited by 7
“These include approaches such as CRNN [35], DTRN [13] and Star-Net [22], that utilize Convolutional Neural Net-14517 works (CNNs) and Recurrent Neural Networks (RNNs), as well as Rosetta [5].”
Open-Set Text Recognition via Shape-Awareness Visual Reconstruction
Liu, C., Yang, C. H., & Yin, X.-C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41731-3_6 · Cited by 7
Xu, M., Zhang, J., Xu, L., Silamu, W., & Li, Y. (2024) · Applied Sciences · doi:10.3390/app14051707 · Cited by 7
CFOR: Character-First Open-Set Text Recognition via Context-Free Learning
Liu, C., Yang, C. H., Fang, Z., Qin, H., & Yin, X.-C. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3480711 · Cited by 7
“Approaches commonly used for this phase include connected components [17], CTC [3], [18], RNN-attention [11], [14], transformer decoder [19], and static queries [16], [20], [21].”
DeepME: Deep Mixture Experts for Large-scale Image Classification
He, M. X., Lv, G., He, W., Fan, J., & Zeng, G. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/100 · Cited by 7
“…in an end-to-end multi-layer manner, deep learning [Hinton et al. , 2006; Jarrett et al. , 2009; Krizhevsky et al. , 2012; Sun et al. , 2014; Borisyuk et al. , 2018; Zhu et al. , 2020; Wang et al. , 2020; Li et al. , 2020] has demonstrated its out-standing performance on significantly…”
BackgroundLi, W., Zhu, H., Yang, S., Wang, P., & Zhang, H. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07617-3 · Cited by 7
“We use ResNet-152 [16] pre-trained on ImageNet as global feature extractor.”
MethodInfluentialMaking the V in Text-VQA Matter
Hegde, S., Jahagirdar, S., & Gangisetty, S. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00590 · Cited by 7
“We consider the M OCR tokens O (generally M is 50) extracted using EasyOCR (for VQA images) and Rosetta [5] (for TextVQA and STVQA images).”
MethodJessen, M., Böschen, F., & Scherp, A. (2019) · ACM Symposium on Document Engineering (DocEng) · doi:10.1145/3342558.3345396 · Cited by 7
“Borisyuk et al. [1] used a state-of-the-art Faster-RCNN object detection network to localize text in natural images.”
Background
Metadata
- Authors
- Borisyuk, F., Gordo, A., & Sivakumar, V.
- Affiliations
- Meta (United States) (US)
- Venue
- Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2018-07-19
- Pages
- 71-79
- OpenAlex
- W2809273748
- Cited by
- 353
- References
- 28
- Citations by year
- 2026: 13 · 2025: 35 · 2024: 44 · 2023: 65 · 2022: 51 · 2021: 79 · 2020: 38 · 2019: 26 · 2018: 2
- FWCI
- 10.92
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Vehicle License Plate Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- optical character recognition, Rosetta, Facebook, system architecture, text extraction from images, media understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.