Rosetta: Large Scale System for Text Detection and Recognition in Images
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
In this paper we present a deployed, scalable optical character recognition (OCR) system, which we call Rosetta , designed to process images uploaded daily at Facebook scale. Sharing of image content has become one of the primary ways to communicate information among internet users within social networks such as Facebook, and the understanding of such media, including its textual information, is of paramount importance to facilitate search and recommendation applications. We present modeling techniques for efficient detection and recognition of text in images and describe Rosetta 's system architecture. We perform extensive evaluation of presented technologies, explain useful practical approaches to build an OCR system at scale, and provide insightful intuitions as to why and how certain components work based on the lessons learnt during the development and deployment of the system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A novel OCR system is developed, named as Rosetta, wherein faster-RCNN is utilized for detecting the text characters from millions of Facebook images, however, fully convolutional CNN is employed for generating the lexicon free transcription of each word [6].”
From Object detection using YOLO: challenges, architectural successors, datasets and applications (2023) · cites this paper as [6]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 3×
“chitectural decision and a natural choice was to use a character-based recognition model. In the initial stages of development, we tested various efficient backbone architectures, including ResNet-18 [17] and SqueezeNet [18]. We evaluated these choices on our synthetic data and ended up selecting ResNet-18 as our main…”
MethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 3×
“Our text detection approach, based on Faster-RCNN model [24], is responsible of detecting regions of the image that contain text.”
MethodInfluentialImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“ns. As we show empirically in Section 5, ShuffleNet is significantly faster than ResNet and did not lead to inferior accuracy. The ShuffleNet convolutional trunk is pre-trained using ImageNet dataset [10]. To train the end-to-end detection system, the model was bootstrapped with an in-house synthetic dataset and then…”
MethodShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982 · cited 1×
“e encoding, RPN, and classifiers) are trained jointly in a supervised, end-to-end manner. Our text detection model uses Faster-RCNN, but replaces the ResNet convolutional body with a ShuffleNet-based [31] architecture for efficiency reasons. As we show empirically in Section 5, ShuffleNet is significantly faster than…”
MethodShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201 · cited 2×
“st of the k fully connected layers that perform character classification in the CHAR model. Orthogonal to this, one can use different body architectures. We experimented with ResNet-18 and SqueezeNet [18], and obtained higher accuracy with ResNet-18 while experiencing only a very small overhead in computation. Image P…”
MethodGraves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“The second approach was based on curriculum learning [6], i.e ., starting with a simpler problem and increasing the difficulty as the model improves.”
BackgroundShi, B., Bai, X., & Yao, C. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2646371 · Cited by 3,172
Edge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656 · cited 1×
“Before the recognition stage, a combination of Edge Boxes [33] and an aggregate channel features (ACF) detector framework [11] is used to detect candidate locations for the words.”
Method
Research works printed but not linked: 1
- [2] Caffe2, 2017. -
Cited by
Items 401–425 of 425
Pandey, A. K., & Shukla, A. K. (2025) · International Conference on Electronics and Computing, Communication Networking Automation Technologies (ICEC2NT) · doi:10.1109/icec2nt65402.2025.11380006
“Finally, real-world systems such as Borisyuk et al. [23] Ro-setta, deployed at-scale by Facebook for text recognition, conceivable reveal production-level CNN-CTC systems which have been found to be limited in effectively working on social media content, especially in cases where overlapping or…”
Scene Chinese Recognition with Mix Attention and LLM
Yang, X., Zheng, H., & Wang, Y. (2025) · IEEE Smart World Congress (SWC) · doi:10.1109/swc65939.2025.00166
“The representative CRNN model uses convolutional networks for feature extraction, followed by bidirectional LSTMs to capture sequential dependencies and optimize using CTC loss in an end-to-end manner [4], [8], [24], [30].”
Seeing the Signs: A Survey of Edge-Deployable OCR Models for Billboard Visibility Analysis
Szankin, M., Venkatasamy, V., & Ying, L. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00630
“…driven by three major families of models: CNN-based, Transformer-based, and large VLMs. Traditional CNN-based approaches such as CRNN [14], Rosetta [4], and TextFuseNet [25] extract spatial features using convolutions and model sequential dependencies with Recurrent Neural Networks (RNNs) or…”
Beyond single image: modified saliency aggregation for universal attacks on STR models
Xu, Y., Chen, P., Xu, J., Xu, W., & Dai, P. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00570-4
Object Detection Using Deep Learning Algorithm in Aircrafts
Bhavana, M., Swapna, N., Vadlamudi, S., Koduru, G. K., Reddy, A. Y., & Rao, K. S. (2025) · Asia Pacific Conference on Innovation in Technology (APCIT) · doi:10.1109/apcit65661.2025.11411188
Smart Retail Monitoring: An End-to-End Transformer-Based System for Shelf Intelligence
Anitha, G., Ryan, J. C., Shashini, V., & Reddy, G. V. N. (2025) · Global AI Summit - International Conference on Artificial Intelligence and Emerging Technology (AI Summit) · doi:10.1109/aisummit66170.2025.11411620
“While many existing works in retail automation have focused on object detection [15] or text recognition [16] as separate tasks, there has been very little progress in building a unified, end-to-end system that handles the entire shelf auditing process.”
Yi, Z., Zhao, J., Lv, J., & Wang, T. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6438441
Kunduru, G. R. (2021) · International Journal of Engineering Research and Science & Technology · doi:10.62643/ijerst.2021.v17.n1.pp126-132
Fu, H., Xie, Z., Jiang, W., Ma, X., & Yang, D. (2026) · Journal of Imaging · doi:10.3390/jimaging12040146
Comparative Analysis of Text Extraction Methods from PDF Files for Subsequent Data Processing
Myasoedov, V. A., & Vishnevskaya, J. A. (2026) · IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering · doi:10.1109/elcon-ee69794.2026.11452828
Resmerita, D. (2022) · theses.fr (ABES) · doi:10.70675/929c89abzf12bz408dz8518z6bae0724a123
Said, N. A. (2026) · IT Professional · doi:10.1109/mitp.2025.3646076
Dewangan, D., & Pandey, R. (2025) · International Journal of Advances in Signal and Image Sciences · doi:10.29284/n0v3dw58
Optical Character Recognition Accuracy on Degraded Documents
Saladaga, M. J., & Talirongan, F. J. B. (2026) · International Journal on Science and Technology · doi:10.71097/ijsat.v17.i2.11198
“Published experiments so far have either been conducted on "un-degraded," like clean material, or on material with old writing styles, leaving testing OCR engines at specified degradation levels as an avenue to be explored (Borisyuk et al., 2018; Springer, 2025).”
EvaHan 2026 Ancient Books Multimodal OCR and Layout Analysis System Technical Report
Zheng, C. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3n7j6t59tuhs
“For instance, a work by Borisyuk et al. (2018) utilized CL in Facebook’s Rosetta system by progressively increasing the difficulty of training data (e.g., text length, deformation degree), which significantly improved the performance and robustness of their sequence recognition models.”
Shen, L. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
MGL-LMM: Generative OCR with Semantic Inpainting for Degraded Mongolian Lead-Type Printed Documents
Luo, Z., Lu, M., Liu, N., Shi, B., 仁庆道尔吉, & Wu, N. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3520-9_46
A Comprehensive Review of Text Information Extraction Techniques for Images and Videos
Kumar, K. (2026) · International Journal of Research and Innovation in Applied Science · doi:10.51584/ijrias.2026.11060276
Chen, Z., Liu, Y., Xie, D., Wang, Z., Yuan, D., & Zhu, H. (2026) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-026-00608-3
FormLens: From Ink to Insight with Adapting Vision-Language Models for Handwritten Form Digitization
Bhattacharyya, S., Mondal, A., & Jawahar, C. V. (2025) · Indian Conference on Computer Vision, Graphics and Image Processing · doi:10.1145/3774521.3774556
Beyond Script Family Boundaries: Towards Unified Open-Set Scene Text Recognition
Liu, C., & Smith, E. H. B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37252-9_14
Shankar, K. V. (2024) · International Journal of Intelligent Systems and Applications in Engineering
Weakly-supervised fingerspelling recognition in British Sign Language videos
Prajwal, K., Bull, H., Momeni, L., Albanie, S., Varol, G., & Zisserman, A. (2022) · Oxford University Research Archive (ORA) (University of Oxford)
Seeing through Occlusion: Instance Tracking for Chart Understanding
Yao, X., Jin, Y., Zhao, X., Bi, X., Wang, Y., & Yu, Y. (2026) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3767308.3836363
TextVQA : A Comprehensive Study of Approaches, Datasets, and Evaluation Metrics
Agrawal, M., Sharma, H., & Srivastava, S. (2026) · Computational Intelligence · doi:10.1111/coin.70310
Metadata
- Authors
- Borisyuk, F., Gordo, A., & Sivakumar, V.
- Affiliations
- Meta (United States) (US)
- Venue
- Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2018-07-19
- Pages
- 71-79
- OpenAlex
- W2809273748
- Cited by
- 353
- References
- 28
- Citations by year
- 2026: 13 · 2025: 35 · 2024: 44 · 2023: 65 · 2022: 51 · 2021: 79 · 2020: 38 · 2019: 26 · 2018: 2
- FWCI
- 10.92
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Vehicle License Plate Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- optical character recognition, Rosetta, Facebook, system architecture, text extraction from images, media understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.