The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Open accessAbstract
Pre-training techniques have been verified successfully in a variety of NLP tasks in recent years. Despite the widespread use of pre-training models for NLP applications, they almost exclusively focus on text-level manipulation, while neglecting layout and style information that is vital for document image understanding. In this paper, we propose the LayoutLM to jointly model interactions between text and layout information across scanned document images, which is beneficial for a great number of real-world document image understanding tasks such as information extraction from scanned documents. Furthermore, we also leverage image features to incorporate words' visual information into LayoutLM. To the best of our knowledge, this is the first time that text and layout are jointly learned in a single framework for document-level pre-training. It achieves new state-of-the-art results in several downstream tasks, including form understanding (from 70.72 to 79.27), receipt understanding (from 94.02 to 95.24) and document image classification (from 93.07 to 94.42). The code and pre-trained LayoutLM models are publicly available at https://aka.ms/layoutlm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Flagged first, then by citations
Research works printed but not linked: 28
- 10.1109/ICDAR.2017.149 · DOI
- Arindam Das , Saikat Roy , and Ujjwal Bhattacharya . 2018 . Document Image Classification with Intra-Domain Transfer Learning and Stacked Generalization of Deep Convolutional Neural Networks. 2018 24th International Conference on Pattern Recognition (ICPR) (2018), 3180--3185. Arindam Das, Saikat Roy, and Ujjwal Bhattacharya. 2018. Document Image Classification with Intra-Domain Transfer Learning and Stacked Generalization of Deep Convolutional Neural Networks. 2018 24th International Conference on Pattern Recognition (ICPR) (2018), 3180--3185.
- Tyler Dauphinee , Nikunj Patel , and Mohammad Mehdi Rashidi . 2019. Modular Multimodal Architecture for Document Classification. ArXiv , Vol. abs/ 1912 .04376 ( 2019 ). Tyler Dauphinee, Nikunj Patel, and Mohammad Mehdi Rashidi. 2019. Modular Multimodal Architecture for Document Classification. ArXiv, Vol. abs/1912.04376 (2019). · arXiv
- Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 4171--4186. https://doi.org/10. 18653/v1/N19--1423 10.18653/v1 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 4171--4186. https://doi.org/10.18653/v1/N19--1423 · DOI
- Jaekyu Ha , Robert M Haralick , and Ihsin T Phillips . 1995 a. Document page decomposition by the bounding-box project . In Proceedings of 3rd International Conference on Document Analysis and Recognition , Vol. 2 . IEEE, 1119--1122. Jaekyu Ha, Robert M Haralick, and Ihsin T Phillips. 1995 a. Document page decomposition by the bounding-box project. In Proceedings of 3rd International Conference on Document Analysis and Recognition, Vol. 2. IEEE, 1119--1122.
- Jaekyu Ha , Robert M Haralick , and Ihsin T Phillips . 1995 b. Recursive XY cut using bounding boxes of connected components . In Proceedings of 3rd International Conference on Document Analysis and Recognition , Vol. 2 . IEEE, 952--955. Jaekyu Ha, Robert M Haralick, and Ihsin T Phillips. 1995 b. Recursive XY cut using bounding boxes of connected components. In Proceedings of 3rd International Conference on Document Analysis and Recognition, Vol. 2. IEEE, 952--955.
- 10.1109/DAS.2016.23 · DOI
- 10.1109/ICDAR.2015.7333910 · DOI
- Kaiming He , Georgia Gkioxari , Piotr Dollá r, and Ross B. Girshick . 2017 . Mask R-CNN. CoRR , Vol . abs/1703.06870 (2017). arxiv: 1703.06870 http://arxiv.org/abs/1703.06870 Kaiming He, Georgia Gkioxari, Piotr Dollá r, and Ross B. Girshick. 2017. Mask R-CNN. CoRR, Vol. abs/1703.06870 (2017). arxiv: 1703.06870 http://arxiv.org/abs/1703.06870 · arXiv
- Guillaume Jaume , Hazim Kemal Ekenel , and Jean-Philippe Thiran . 2019 . FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents. 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW) , Vol. 2 (2019), 1--6. Guillaume Jaume, Hazim Kemal Ekenel, and Jean-Philippe Thiran. 2019. FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents. 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), Vol. 2 (2019), 1--6.
- 10.18653/v1/D18-1476 · DOI
- Ranjay Krishna , Yuke Zhu , Oliver Groth , Justin Johnson , Kenji Hata , Joshua Kravitz , Stephanie Chen , Yannis Kalantidis , Li-Jia Li , David A Shamma , Michael Bernstein , and Li Fei-Fei . 2016 . Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations. https://arxiv.org/abs/1602.07332 Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, Michael Bernstein, and Li Fei-Fei. 2016. Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations. https://arxiv.org/abs/1602.07332 · arXiv
- 10.1109/ICPR.1992.201771 · DOI
- D. Lewis , G. Agam , S. Argamon , O. Frieder , D. Grossman , and J. Heard . 2006. Building a Test Collection for Complex Document Information Processing . In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval ( Seattle, Washington, USA) (SIGIR '06). ACM, New York, NY, USA, 665--666. https://doi.org/10.1145/1148170.1148307 10.1145/1148170.1148307 D. Lewis, G. Agam, S. Argamon, O. Frieder, D. Grossman, and J. Heard. 2006. Building a Test Collection for Complex Document Information Processing. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Seattle, Washington, USA) (SIGIR '06). ACM, New York, NY, USA, 665--666. https://doi.org/10.1145/1148170.1148307 · DOI
- 10.18653/v1/N19-2005 · DOI
- Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke S. Zettlemoyer , and Veselin Stoyanov . 2019 b. RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv , Vol. abs/ 1907 .11692 ( 2019 ). Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke S. Zettlemoyer, and Veselin Stoyanov. 2019 b. RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv, Vol. abs/1907.11692 (2019). · arXiv
- 10.1109/TPAMI.2005.4 · DOI
- 10.1109/34.244677 · DOI
- 10.24963/ijcai.2019/466 · DOI
- Sebastian Schreiber , Stefan Agne , Ivo Wolf , Andreas Dengel , and Sheraz Ahmed . 2017 . DeepDeSRT: Deep Learning for Detection and Structure Recognition of Tables in Document Images. 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) , Vol. 01 (2017), 1162--1167. Sebastian Schreiber, Stefan Agne, Ivo Wolf, Andreas Dengel, and Sheraz Ahmed. 2017. DeepDeSRT: Deep Learning for Detection and Structure Recognition of Tables in Document Images. 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Vol. 01 (2017), 1162--1167.
- 10.1109/ICCV.2005.140 · DOI
- 10.1109/34.584106 · DOI
- Carlos Soto and Shinjae Yoo . 2019. Visual Detection with Context for Document Layout Analysis . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Association for Computational Linguistics , Hong Kong , China, 3462--3468. https://doi.org/10. 1865 3/v1/D19--1348 10.18653/v1 Carlos Soto and Shinjae Yoo. 2019. Visual Detection with Context for Document Layout Analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 3462--3468. https://doi.org/10.18653/v1/D19--1348 · DOI
- Christian Szegedy Sergey Ioffe Vincent Vanhoucke and Alex Alemi. 2016. Inception-v4 Inception-ResNet and the Impact of Residual Connections on Learning. In AAAI . Christian Szegedy Sergey Ioffe Vincent Vanhoucke and Alex Alemi. 2016. Inception-v4 Inception-ResNet and the Impact of Residual Connections on Learning. In AAAI . · DOI · Paper page
- Matheus Palhares Viana and Dário Augusto Borges Oliveira . 2017 . Fast CNN-Based Document Layout Analysis. 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) (2017), 1173--1180. Matheus Palhares Viana and Dário Augusto Borges Oliveira. 2017. Fast CNN-Based Document Layout Analysis. 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) (2017), 1173--1180.
- H. Wei , M. Baechler , F. Slimane , and R. Ingold . 2013. Evaluation of SVM , MLP and GMM Classifiers for Layout Analysis of Historical Documents. In 2013 12th International Conference on Document Analysis and Recognition. 1220--1224 . https://doi.org/10.1109/ICDAR. 2013 .247 10.1109/ICDAR.2013.247 H. Wei, M. Baechler, F. Slimane, and R. Ingold. 2013. Evaluation of SVM, MLP and GMM Classifiers for Layout Analysis of Historical Documents. In 2013 12th International Conference on Document Analysis and Recognition. 1220--1224. https://doi.org/10.1109/ICDAR.2013.247 · DOI
- 10.1109/CVPR.2017.462 · DOI
- Xu Zhong , Jianbin Tang , and Antonio Jimeno-Yepes . 2019. PubLayNet: largest dataset ever for document layout analysis. ArXiv , Vol. abs/ 1908 .07836 ( 2019 ). Xu Zhong, Jianbin Tang, and Antonio Jimeno-Yepes. 2019. PubLayNet: largest dataset ever for document layout analysis. ArXiv, Vol. abs/1908.07836 (2019). · arXiv
Cited by
Sorted by citations
Peña, A., Morales, A., Fierrez, J., Ortega-Garcia, J., Puente, I., Cordova, J., & Cordova, G. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102398 · Cited by 16
olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models
Poznanski, J., Rangapur, A., Borchardt, J., Dunkelberger, J., Huff, R., Lin, D., Rangapur, A., Wilhelm, C., Lo, K., & Soldaini, L. (2025) · arXiv · doi:10.48550/arxiv.2502.18443 · Cited by 5
Shanivendra, A. C. (2026) · arXiv · doi:10.48550/arxiv.2603.14170
Metadata
- Authors
- Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., & Zhou, M.
- Affiliations
- Harbin Institute of Technology (CN)
- Beihang University (CN)
- Microsoft Research Asia (China) (CN)
- Venue
- Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2020-08-20
- Pages
- 1192-1200
- OpenAlex
- W2997154779
- Cited by
- 812
- References
- 2
- Citations by year
- 2026: 212 · 2025: 200 · 2024: 132 · 2023: 133 · 2022: 77 · 2021: 51 · 2020: 7
- FWCI
- 20.36
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- document image understanding, scanned documents, information extraction from documents, document image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
