The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Lipreading With Local Spatiotemporal Descriptors
Abstract
Visual speech information plays an important role in lipreading under noisy conditions or for listeners with a hearing impairment. In this paper, we present local spatiotemporal descriptors to represent and recognize spoken isolated phrases based solely on visual input. Spatiotemporal local binary patterns extracted from mouth regions are used for describing isolated phrase sequences. In our experiments with 817 sequences from ten phrases and 20 speakers, promising accuracies of 62% and 70% were obtained in speaker-independent and speaker-dependent recognition, respectively. In comparison with other methods on AVLetters database, the accuracy, 62.8%, of our method clearly outperforms the others. Analysis of the confusion matrix for 26 English letters shows the good clustering characteristics of visemes for the proposed descriptors. The advantages of our approach include local processing and robustness to monotonic gray-scale changes. Moreover, no error prone segmentation of moving lips is needed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
Ojala, T., Pietikainen, M., & Maenpaa, T. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017623 · Cited by 15,398
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Face Description with Local Binary Patterns: Application to Face Recognition
Ahonen, T., Hadid, A., & Pietikainen, M. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.244 · Cited by 5,584
Self-citationDynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
Self-citationXM2VTSDB: The Extended M2VTS Database
Messer, K., Matas, J., Kittler, J., Luettin, J., & Maître, G. (1999) · Cited by 1,158
Recent advances in the automatic recognition of audiovisual speech
Pomianos, G., Neti, C., Gravier, G., Garg, A., & Senior, A. W. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817150 · Cited by 752
Extraction of visual features for lipreading
Matthews, I., Cootes, T. F., Bangham, J. A., Cox, S., & Harvey, R. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.982900 · Cited by 537
Modeling Coarticulation in Synthetic Visual Speech
Cohen, M. M., & Massaro, D. W. (1993) · Models and Techniques in Computer Animation · doi:10.1007/978-4-431-66911-1_13 · Cited by 445
BiolD: a multimodal biometric identification system
Frischholz, R. W., & Dieckmann, U. (2000) · Computer · doi:10.1109/2.820041 · Cited by 346
Cited by
(2011) · Cited by 2,752
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54184-6_6 · Cited by 537
Temporal Multimodal Learning in Audiovisual Speech Recognition
Hu, D., Li, X., & Lu, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.389 · Cited by 108
Zhou, H., Du, J., Zhang, Y., Wang, Q., Liu, Q.-F., & Lee, C.-H. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3096037 · Cited by 85
Exploring ROI size in deep learning based lipreading
Koumparoulis, A., Potamianos, G., Mroueh, Y., & Rennie, S. J. (2017) · The 14th International Conference on Auditory-Visual Speech Processing · doi:10.21437/avsp.2017-13 · Cited by 21
Metadata
- Authors
- Zhao, G., Barnard, M., & Pietikainen, M.
- Venue
- IEEE Transactions on Multimedia
- Published
- 2009-11
- Volume
- 11
- Issue
- 7
- Pages
- 1254-1265
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2136155248
- Cited by
- 309
- References
- 32
- FWCI
- 14.97
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Lipreading With Local Spatiotemporal Descriptors. https://app.cochat.ai/papers/10.1109/tmm.2009.2030637
- The paper
- Zhao, G., Barnard, M., & Pietikainen, M. (2009). Lipreading With Local Spatiotemporal Descriptors. IEEE Transactions on Multimedia. https://doi.org/10.1109/tmm.2009.2030637
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
