CoChat
Sign Up
PreprintCited by 6Handwritten Text Recognition TechniquesMachine Learning and AlgorithmsText and Document Classification Technologies

OLALA: Object-Level Active Learning for Efficient Document Layout Annotation

Open access
Shen, Zejiang, Zhao, Jian, Dell, Melissa, Yu, Yaoliang, Li, Weining·2020·arXiv
DOI 10.48550/arxiv.2010.01762arXiv 2010.01762
Open this paper in CoChat — free accountOpen PDF

Abstract

Document images often have intricate layout structures, with numerous content regions (e.g. texts, figures, tables) densely arranged on each page. This makes the manual annotation of layout datasets expensive and inefficient. These characteristics also challenge existing active learning methods, as image-level scoring and selection suffer from the overexposure of common objects.Inspired by recent progresses in semi-supervised learning and self-training, we propose an Object-Level Active Learning framework for efficient document layout Annotation, OLALA. In this framework, only regions with the most ambiguous object predictions within an image are selected for annotators to label, optimizing the use of the annotation budget. For unselected predictions, the semi-automatic correction algorithm is proposed to identify certain errors based on prior knowledge of layout structures and rectifies them with minor supervision. Additionally, we carefully design a perturbation-based object scoring function for document images. It governs the object selection process via evaluating prediction ambiguities, and considers both the positions and categories of predicted layout objects. Extensive experiments show that OLALA can significantly boost model performance and improve annotation efficiency, given the same labeling budget. Code for this paper can be accessed via https://github.com/lolipopshock/detectron2_al.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2010.01762

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Other applications such as document layout detection, pose estimation, and surface defect detection remain under-explored.

    “Other applications such as document layout detection [14,17], pose estimation [7], and surface defect detection [55] remain under-explored, as well.”

    From Ten Years of Active Learning Techniques and Object Detection: A Systematic Review (2023) · cites this paper as [14]

    The Abstract and Introduction identify document layout detection as the application of OLALA and explain why existing image-level active-learning methods are poorly suited to densely arranged document regions. They do not establish that document layout detection generally remains under-explored. This citation is attached to document layout detection, not to pose estimation or surface defect detection. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • The most advanced companies and research centres in AI are interested in continuing research around active learning and exploring its potential benefits further.

    “On the other hand, the most advanced companies and research centres in the field of AI are also interested in continuing research around AL and exploring its potential benefits further [14,36,50,59,78].”

    From Ten Years of Active Learning Techniques and Object Detection: A Systematic Review (2023) · cites this paper as [14]

    The Introduction (page unavailable) proposes a new active-learning framework, and the Conclusion (page unavailable) describes its aim of improving document-annotation efficiency. This supports the paper as an example of continued research into AL’s benefits, but it does not establish that the most advanced AI companies and research centres are interested in such research. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 23

  1. Apostolos Antonacopoulos, David Bridson, Christos Papadopoulos, and Stefan Pletschacher. 2009. A real- istic dataset for performance evaluation of document layout analysis. In 2009 10th International Confer- ence on Document Analysis and Recognition, pages 296-300. IEEE.
  2. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international confer- ence on machine learning, pages 41-48.
  3. Clemens-Alexander Brust, Christoph Käding, and Joachim Denzler. 2018. Active learning for deep object detection. arXiv preprint arXiv:1809.09875. · arXiv
  4. Sai Vikas Desai, Akshay Chandra Lagandula, Wei Guo, Seishi Ninomiya, and Vineeth N Balasubramanian. 2019. An adaptive supervision framework for ac- tive learning in object detection. arXiv preprint arXiv:1908.02454. · arXiv
  5. Priya Goyal, Piotr Dollár, Ross Girshick, Pieter No- ordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. 2017. Ac- curate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677. · arXiv
  6. Elmar Haussmann, Michele Fenzi, Kashyap Chitta, Jan Ivanecky, Hanson Xu, Donna Roy, Akshita Mittel, Nicolas Koumchatzky, Clement Farabet, and Jose M Alvarez. 2020. Scalable active learning for object detection. arXiv preprint arXiv:2004.04699. · arXiv
  7. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770- 778.
  8. Zhuoren Jiang, Zhe Gao, Yuguang Duan, Yangyang Kang, Changlong Sun, Qiong Zhang, and Xiaozhong Liu. 2020. Camouflaged chinese spam content de- tection with semi-supervised generative active learn- ing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3080-3085.
  9. Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou, and Zhoujun Li. 2019. Tablebank: Ta- ble benchmark for image-based table detection and recognition. arXiv preprint arXiv:1903.01949. · arXiv
  10. Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117-2125.
  11. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European confer- ence on computer vision, pages 740-755. Springer.
  12. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object de- tection with region proposal networks. In Advances in neural information processing systems, pages 91- 99.
  13. Soumya Roy, Asim Unmesh, and Vinay P Namboodiri. 2018. Deep active learning for object detection. In BMVC, page 91.
  14. Sebastian Schreiber, Stefan Agne, Ivo Wolf, Andreas Dengel, and Sheraz Ahmed. 2017. Deepdesrt: Deep learning for detection and structure recognition of tables in document images. In 2017 14th IAPR in- ternational conference on document analysis and recognition (ICDAR), volume 1, pages 1162-1167. IEEE.
  15. H Sebastian Seung, Manfred Opper, and Haim Som- polinsky. 1992. Query by committee. In Proceed- ings of the fifth annual workshop on Computational learning theory, pages 287-294.
  16. Zejiang Shen, Kaixuan Zhang, and Melissa Dell. 2020. A large dataset of historical japanese documents with complex layouts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition Workshops, pages 548-549.
  17. Zejiang Shen, Ruochen Zhang, Melissa Dell, Benjamin Charles Germain Lee, Jacob Carlson, and Weining Li. 2021. Layoutparser: A unified toolkit for deep learn- ing based document image analysis. In International Conference on Document Analysis and Recognition, pages 131-146. Springer.
  18. Ilya Sutskever, James Martens, George Dahl, and Ge- offrey Hinton. 2013. On the importance of initial- ization and momentum in deep learning. In Interna- tional conference on machine learning, pages 1139- 1147.
  19. Keze Wang, Xiaopeng Yan, Dongyu Zhang, Lei Zhang, and Liang Lin. 2018. Towards human-machine co- operation: Self-supervised sample mining for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1605-1613.
  20. Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. 2020. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10687-10698.
  21. Angela Yao, Juergen Gall, Christian Leistner, and Luc Van Gool. 2012. Interactive object detection. In 2012 IEEE conference on computer vision and pattern recognition, pages 3242-3249. IEEE.
  22. Xu Zhong, Jianbin Tang, and Antonio Jimeno Yepes. 2019. Publaynet: largest dataset ever for document layout analysis. arXiv preprint arXiv:1908.07836. · arXiv
  23. Zongwei Zhou, Jae Shin, Lei Zhang, Suryakanth Gurudu, Michael Gotway, and Jianming Liang. 2017. Fine-tuning convolutional neural networks for biomedical image analysis: actively and incre- mentally. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7340-7351.

Cited by

Sorted by citations

Authors
Shen, Z., Zhao, J., Dell, M., Yu, Y., & Li, W.
Affiliations
Harvard University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
Pages
170-182
DOI
10.48550/arxiv.2010.01762
arXiv
2010.01762
OpenAlex
W3146762582
Cited by
6
References
1
Citations by year
2026: 1 · 2025: 1 · 2024: 1 · 2023: 1 · 2021: 2
FWCI
0.27
Type
conference paper
Language
en
Open access
Licence
Locations
  • Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (1823616)
  • Harvard Catalyst
  • Harvard Data Science Initiative, Harvard University
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
  • Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
Keywords
annotation efficiency, historical document images, semi-supervised learning, self-training

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review