Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 32

The DOI matches its Crossref record.

End-to-End page-Level assessment of handwritten text recognition

Vidal, Enrique, Toselli, Alejandro H., Ríos-Vila, Antonio, Calvo-Zaragoza, Jorge·2023·Pattern Recognition
DOI 10.1016/j.patcog.2023.109695
Open this paper in CoChat — free account

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.patcog.2023.109695

Cites

Statistics

Figures

Figure 1 Example of misleading WER evaluation, caused by wrong reading order due to text-line detection flaws. While all the words in the automatic transcript are perfectly correct, the WER is 70% (13 matching words, 13 substitution errors, 4 insertions and 4 deletions). The image is part of a page of the Bentham Papers collection (see Sec. 3).
Figure 1 Example of misleading WER evaluation, caused by wrong reading order due to text-line detection flaws. While all the words in the automatic transcript are perfectly correct, the WER is 70% (13 matching words, 13 substitution errors, 4 insertions and 4 deletions). The image is part of a page of the Bentham Papers collection (see Sec. 3).CC-BY-NC-ND · From the PDF
Figure 2 Examples of frequent reading order issues, from the Bentham Papers collection.
Figure 2 Examples of frequent reading order issues, from the Bentham Papers collection.CC-BY-NC-ND · From the PDF
Figure 3 Assignment obtained by the Hungarian algorithm for a bipartite graph corresponding to the word sequences X and Y . Dotted edges have a null cost and coloured words are insertions or deletions. For this assignment, d h ( X,Y ) = 1 + 1 + 4 / 2 + 2 / 2 + 1 + 1 + 3 / 2 = 8 . 5 .
Figure 3 Assignment obtained by the Hungarian algorithm for a bipartite graph corresponding to the word sequences X and Y . Dotted edges have a null cost and coloured words are insertions or deletions. For this assignment, d h ( X,Y ) = 1 + 1 + 4 / 2 + 2 / 2 + 1 + 1 + 3 / 2 = 8 . 5 .CC-BY-NC-ND · From the PDF
Figure 4 Page images of the ICFHR14 dataset.
Figure 4 Page images of the ICFHR14 dataset.CC-BY-NC-ND · From the PDF
Figure 5 Evaluation results for increasingly distorted transcripts, as a function of the CER artificially induced by the distortion process. Left: words are distorted individually avoiding induced white space errors to break or join words. Right: distortion is applied at full line level, allowing white space to be deleted/inserted between/within words. Curves with very similar or identical values are depicted with the same colour and/or point shape. The prefix 't' in tCER, tWER and tNSFD indicates the corresponding values are theoretically computed.
Figure 5 Evaluation results for increasingly distorted transcripts, as a function of the CER artificially induced by the distortion process. Left: words are distorted individually avoiding induced white space errors to break or join words. Right: distortion is applied at full line level, allowing white space to be deleted/inserted between/within words. Curves with very similar or identical values are depicted with the same colour and/or point shape. The prefix 't' in tCER, tWER and tNSFD indicates the corresponding values are theoretically computed.CC-BY-NC-ND · From the PDF
Figure 6 Evaluation results on actual HTR transcripts where the line order is distorted by random line swaps and breaks. Curves with almost identical values are depicted with the same colour and symbol. tNSFD corresponds to theoretically computed values. The 'swp' and 'brk' labels denote line swap and split, resp. (see Sec. 7.4 and 7.5).
Figure 6 Evaluation results on actual HTR transcripts where the line order is distorted by random line swaps and breaks. Curves with almost identical values are depicted with the same colour and symbol. tNSFD corresponds to theoretically computed values. The 'swp' and 'brk' labels denote line swap and split, resp. (see Sec. 7.4 and 7.5).CC-BY-NC-ND · From the PDF
Figure 7 Real examples from Bentham Papers images 010 003 002, 019 004 003, which illustrate text line splitting errors that affect RO. In the simulation experiments these examples correspond, top to bottom, to Cases 1, 2 and 3.
Figure 7 Real examples from Bentham Papers images 010 003 002, 019 004 003, which illustrate text line splitting errors that affect RO. In the simulation experiments these examples correspond, top to bottom, to Cases 1, 2 and 3.CC-BY-NC-ND · From the PDF
Figure 8 Correlation of WER -bWER (and WER -hWER ) with NSFD. Real results from Table 3 are included, along with a straight line fitted to these points. Curves with almost identical values are depicted with the same colour and symbol.
Figure 8 Correlation of WER -bWER (and WER -hWER ) with NSFD. Real results from Table 3 are included, along with a straight line fitted to these points. Curves with almost identical values are depicted with the same colour and symbol.CC-BY-NC-ND · From the PDF
Figure 9 Computing Times of hWER , O ( N 3 ) , WER , O ( N 2 ) and bWER , O ( N ) , fitted respectively to polynomials of degrees 3, 2 and 1 (linear).
Figure 9 Computing Times of hWER , O ( N 3 ) , WER , O ( N 2 ) and bWER , O ( N ) , fitted respectively to polynomials of degrees 3, 2 and 1 (linear).CC-BY-NC-ND · From the PDF
Image 10
CC-BY-NC-ND · From the PDF
Image 11
CC-BY-NC-ND · From the PDF
Image 12
CC-BY-NC-ND · From the PDF
Image 13
CC-BY-NC-ND · From the PDF
Image 14
CC-BY-NC-ND · From the PDF
Image 15
CC-BY-NC-ND · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 15

  1. Clausner, 2020, Flexible character accuracy measure for reading-order-independent evaluation, Pattern Recognit. Lett., 131, 390, 10.1016/j.patrec.2020.02.003 · DOI
  2. Kendall, 1938, A new measure of rank correlation, Biometrika, 30, 81, 10.2307/2332226 · DOI
  3. Kuhn, 1955, The hungarian method for the assignment problem, Naval Res. Logist. Quarter., 2, 83, 10.1002/nav.3800020109 · DOI · Paper page
  4. C.-Y. Lee, C.-L. Li, C. Wang, R. Wang, Y. Fujii, S. Qin, A. Popat, T. Pfister, Rope: reading order equivariant positional encoding for graph-based document information extraction, 2021, (????). ArXiv preprint arXiv:2106.10786. · DOI
  5. Liang, 2019, Reassembling shredded document stripes using word-path metric and greedy composition optimal matching solver, IEEE Trans. Multimed., 22, 1168, 10.1109/TMM.2019.2941777 · DOI
  6. Liu, 2002, Optimal matching problem in detection and recognition performance evaluation, Pattern Recognit., 35, 2125, 10.1016/S0031-3203(01)00204-7 · DOI
  7. Marti, 2002, The IAM-database: an english sentence database for offline handwriting recognition, Int. J. Doc. Anal. Recogn., 5, 39, 10.1007/s100320200071 · DOI
  8. Muehlberger, 2019, Transforming scholarship in the archives through handwritten text recognition: transkribus as a case study, J. Document., 10.1108/JD-07-2018-0114 · DOI · Paper page
  9. Ney, 2000, Algorithms for statistical translation of spoken language, IEEE Trans. Speech Audio Process., 8, 24, 10.1109/89.817451 · DOI
  10. Popović, 2011, Towards automatic error analysis of machine translation output, Comput. Linguist., 37, 657, 10.1162/COLI_a_00072 · DOI
  11. L. Quirós, E. Vidal, Evaluation of a region proposal architecture for multi-task document layout analysis, 2021, (????). CoRR. arXiv: 2106.11797 · arXiv
  12. Sánchez, 2019, A set of benchmarks for handwritten text recognition on historical documents, Pattern Recognit, 94, 122, 10.1016/j.patcog.2019.05.025 · DOI
  13. A.H. Toselli, E. Vidal, Revisiting Bag-of-Word metrics to assess end-to-end text image recognition results, 2023, (????). To be published. · DOI
  14. Wagner, 1974, The string-to-string correction problem, J. ACM (JACM), 21, 168, 10.1145/321796.321811 · DOI
  15. Yin, 2009, Handwritten chinese text line segmentation by clustering with distance metric learning, Pattern Recognit., 42, 3146, 10.1016/j.patcog.2008.12.013 · DOI

Cited by

Metadata

Authors
Vidal, E., Toselli, A. H., Ríos-Vila, A., & Calvo-Zaragoza, J.
ORCID
Vidal, Enrique
Venue
Pattern Recognition
Publisher
Elsevier BV
Published
2023-10
Volume
142
Pages
109695
ISSN
0031-3203
DOI
10.1016/j.patcog.2023.109695
Cited by
32
References
0
Language
en
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.