Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 250

The DOI matches its Crossref record.

TCD-TIMIT: An Audio-Visual Corpus of Continuous Speech

Harte, Naomi, Gillen, Eoin · 2015 · IEEE Transactions on Multimedia

Abstract

Automatic audio-visual speech recognition currently lags behind its audio-only counterpart in terms of major progress. One of the reasons commonly cited by researchers is the scarcity of suitable research corpora. This paper details the creation of a new corpus designed for continuous audio-visual speech recognition research . TCD-TIMIT consists of high-quality audio and video footage of 62 speakers reading a total of 6913 phonetically rich sentences. Three of the speakers are professionally-trained lipspeakers, recorded to test the hypothesis that lipspeakers may have an advantage over regular speakers in automatic visual speech recognition systems. Video footage was recorded from two angles: straight on, and at 30°. The paper outlines the recording of footage, and the required post-processing to yield video and audio clips for each sentence. Audio, visual, and joint audio-visual baseline experiments are reported. Separate experiments were run on the lipspeaker and non-lipspeaker data, and the results compared. Visual and audio-visual baseline results on the non-lipspeakers were low overall. Results on the lipspeakers were found to be significantly higher. It is hoped that as a publicly available database, TCD-TIMIT will now help further state of the art in audio-visual speech recognition research.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tmm.2015.2407694

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 54

  1. hosom, 2000, Automatic time alignment of phonemes using acoustic-phonetic information
  2. vertanen, 2006, ?Baseline WSJ acoustic models for HTK and Sphinx Training recipes and recognition experiments ?
  3. 10.1109/MMSP.2006.285261 · DOI
  4. saitoh, 2010, A study of influence of word lip reading by change of frame rate, Proc AVSP-2010, 131
  5. garofolo, 1993, DARPA TIMIT Acoustic Phonetic Continuous Speech Corpus CDROM
  6. burnham, 2009, A blueprint for a comprehensive Australian English auditory-visual speech corpus, Sel Proc 2008 HCSNet Workshop Designing Aust Nat Corpus, 96
  7. sj�lander, 2003, An hmm-based system for automatic segmentation and alignment of speech, Proc Fonetik, 93
  8. 10.1016/S0167-6393(98)00031-4 · DOI
  9. 10.1016/0167-6393(93)90083-W · DOI · Paper page
  10. 2010, Algorithms to Measure Audio Programme Loudness and True-peak Audio Level
  11. galatas, 2012, Audiovisual speech recognition incorporating facial depth information captured by the Kinect, Proc 20th Eur Signal Process Conf, 2714
  12. 10.1109/ICME.2012.192 · DOI
  13. 10.1016/j.patrec.2011.06.011 · DOI
  14. 10.1109/JPROC.2003.817150 · DOI · Paper page
  15. petajan, 1984, Automatic lipreading to enhance speech recognition (speech reading)
  16. 10.1121/1.2229005 · DOI · Paper page
  17. sanderson, 2008, Biometric Person Recognition Face Speech and Fusion
  18. 10.1109/ICME.2001.1237849 · doi:10.1109/icme.2001.1237849
  19. movellan, 1995, Advances in neural information processing systems, 7, 851
  20. fox, 2005, Audio-and Video-Based Biometric Person Authentication
  21. 10.1109/ICASSP.2007.366941 · doi:10.1109/icassp.2007.366941
  22. messer, 1999, XM2VTSDB: The extended M2VTS database, Proc 2nd Int Conf Audio- Video-Based Biometric Person Authentication, 72
  23. 10.1109/ICASSP.2004.1326155 · doi:10.1109/icassp.2004.1326155
  24. 10.1145/2141622.2141646 · doi:10.1145/2141622.2141646
  25. 10.1109/ICASSP.2014.6854158 · DOI
  26. 10.1109/ICASSP.2001.940793 · DOI
  27. 10.1109/TASL.2008.2011515 · DOI
  28. lamel, 1986, Speech database development: Design and analysis of the acoustic-phonetic corpus, Proc DARPA Speech Recog Workshop, 100
  29. driel, 2009, Building a visual speech recognizer
  30. 10.1121/1.2935783 · DOI · Paper page
  31. 10.2991/jcis.2008.61 · doi:10.2991/jcis.2008.61
  32. chen, 2001, Audiovisual speech processing, IEEE Signal Process Mag, 18, 9, 10.1109/79.911195 · DOI · Paper page
  33. 10.1109/34.982900 · DOI · Paper page
  34. 10.1145/1027933.1027972 · DOI
  35. kleinschmidt, 2007, A continuous speech recognition evaluation protocol for the AVICAR database, Proc Int Conf Signal Process Commun, 339
  36. bailly-baillire, 2003, The BANCA database and evaluation protocol, Proc Int Conf Audio- Video-Based Biometric Person Authentication, 625, 10.1007/3-540-44887-X_74 · DOI · Paper page
  37. 10.1109/ICASSP.2002.1006168 · doi:10.1109/icassp.2002.1006168
  38. chibelushi, 1996, design issues for a digital audio-visual integrated database, IEE Colloquium on Integrated Audio-Visual Processing for Recognition Synthesis and Communication (Ref No 1996/213), 7/1
  39. goecke, 2004, A detailed description of the Avozes data corpus, Proc 10th Australian Int Conf Speech Sci Technol, 486
  40. 10.1109/6046.985551 · DOI · Paper page
  41. gan, 2012, Bimodal speech recognition
  42. sui, 2012, A 3D audio-visual corpus for speech recognition, Proc SSST, 125
  43. 10.1109/ICIP.2010.5650963 · doi:10.1109/icip.2010.5650963
  44. cappelletta, 2012, What is a viseme? Exploring the visual side of automatic speech recognition
  45. zhang, 2002, Automatic speechreading with applications to human-computer interfaces, EURASIP J Appl Signal Process, 2002, 1228, 10.1155/S1110865702206137 · DOI
  46. chitu, 2007, Building a data corpus for audio-visual speech recognition, Proc Euromedia 2007, 88
  47. 10.1155/2008/810362 · DOI
  48. heckmann, 2002, DCT-based video features for audio-visual speech recognition, Proc INTERSPEECH, 1925
  49. neti, 2000, ?Audio-visual speech recognition ?
  50. scanlon, 2003, Visual feature analysis for automatic speechreading, Proc Int Conf Audio-Visual Speech Process, 127
  51. bliss, 1972, Languages in contact: Some problems of hiberno-english, Proc Royal Irish Academy Section C Archaeol Celtic Studies History Linguistics Literature, 63
  52. 10.1109/29.46546 · DOI · Paper page
  53. young, 2006, The HTK Book Version 3 4
  54. jeffers, 1971, Speechreading (Lipreading)

Cited by

See all 250

Metadata

Authors
Harte, N., & Gillen, E.
Venue
IEEE Transactions on Multimedia
Published
2015-05
Volume
17
Issue
5
Pages
603-615
ISSN
  • 1520-9210 (ISSN-L)
  • 1941-0077
OpenAlex
W2029199293
Cited by
250
References
0
FWCI
16.81
Type
journal article
Language
en
Open access
closed
Funders
Science Foundation Ireland (07/EN/E007)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: TCD-TIMIT: An Audio-Visual Corpus of Continuous Speech. https://app.cochat.ai/papers/10.1109/tmm.2015.2407694
The paper
Harte, N., & Gillen, E. (2015). TCD-TIMIT: An Audio-Visual Corpus of Continuous Speech. IEEE Transactions on Multimedia. https://doi.org/10.1109/tmm.2015.2407694

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.