CoChat

Foreign accent conversion in computer assisted pronunciation training

Open access
Felps, Daniel, Bortfeld, Heather, Gutierrez-Osuna, Ricardo · 2009 · Speech Communication
Cited by 130

The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.

Abstract

Learners of a second language practice their pronunciation by listening to and imitating utterances from native speakers. Recent research has shown that choosing a well-matched native speaker to imitate can have a positive impact on pronunciation training. Here we propose a voice-transformation technique that can be used to generate the (arguably) ideal voice to imitate: the own voice of the learner with a native accent. Our work extends previous research, which suggests that providing learners with prosodically corrected versions of their utterances can be a suitable form of feedback in computer assisted pronunciation training. Our technique provides a conversion of both prosodic and segmental characteristics by means of a pitch-synchronous decomposition of speech into glottal excitation and spectral envelope. We apply the technique to a corpus containing parallel recordings of foreign-accented and native-accented utterances, and validate the resulting accent conversions through a series of perceptual experiments. Our results indicate that the technique can reduce foreign accentedness without significantly altering the voice quality properties of the foreign speaker. Finally, we propose a pedagogical strategy for integrating accent conversion as a form of behavioral shaping in computer assisted pronunciation training.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.specom.2008.11.004

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 56

  1. Abe, M., Nakamura, S., Shikano, K., Kuwabara, H., 1988. Voice conversion through vector quantization. In: Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing, New York, NY, pp. 655–658. · DOI · Paper page
  2. Anisfeld, 1962, Evaluational reactions to accented English speech, J. Abnorm. Social Psychol., 65, 223, 10.1037/h0045060 · DOI
  3. Arslan, 1997, A study of temporal features and frequency characteristics in American English foreign accent, J. Acoust. Soc. Amer., 102, 28, 10.1121/1.419608 · DOI
  4. Arslan, L.M., Talkin, D., 1997. Voice Conversion By Codebook Mapping Of Line Spectral Frequencies And Excitation Spectrum, Eurospeech’97, Rhodes, Greece, pp. 1347–1350. · DOI
  5. Arthur, 1974, Evaluation reactions of college students to dialect differences in the English of Mexican–Americans, Language Speech, 17, 255, 10.1177/002383097401700304 · doi:10.1177/002383097401700304
  6. Childers, 1989, Voice conversion, Speech Commun., 8, 147, 10.1016/0167-6393(89)90041-1 · DOI
  7. Chun, 1998, Signal analysis software for teaching discourse intonation, Language Learn. Technol., 2, 61, 10.64152/10125/25033 · DOI
  8. Compton, 1963, Effects of filtering and vocal duration upon identification of speakers, aurally, J. Acoust. Soc. Amer., 35, 1748, 10.1121/1.1918810 · DOI
  9. Derwing, 1998, Evidence in favor of a broad framework for pronunciation instruction, Language Learn., 48, 393, 10.1111/0023-8333.00047 · DOI · Paper page
  10. Derwing, 1998, Evidence in favour of a broad framework for pronunciation instruction, Language Learn., 48, 393, 10.1111/0023-8333.00047 · DOI · Paper page
  11. Eskenazi, 1999, Using automatic speech processing for foreign language pronunciation tutoring: some issues and a prototype, Language Learn. Technol., 2, 62, 10.64152/10125/25043 · DOI · Paper page
  12. Eskenazi, M., Hansma, S., 1998. The fluency pronunciation trainer. In: Proc. STiLL Workshop on Speech Technology in Language Learning. · doi:10.21437/eurospeech.1999-219
  13. Griffin, 1984, Signal estimation from modified short-time Fourier transform, IEEE Trans. Acoust. Speech Signal Process., 32, 236, 10.1109/TASSP.1984.1164317 · DOI · Paper page
  14. Hansen, T.K., 2006. Computer assisted pronunciation training: the four ‘K’s of feedback. In: 4th Internat. Conf. on Multimedia and Information and Communication Technologies in Education, Seville, Spain, pp. 342–346.
  15. Hincks, 2003, Speech technologies for pronunciation feedback and evaluation, ReCALL, 15, 3, 10.1017/S0958344003000211 · DOI
  16. Huckvale, M., Yanagisawa, K., 2007. Spoken language conversion with accent morphing. In: Proc. ISCA Speech Synthesis Workshop, Bonn, Germany, pp. 64–70.
  17. Jilka, M., Möhler, G., 1998. Intonational foreign accent: speech technology and foreign language teaching. In: Proc. ESCA Workshop on Speech Technology in Language Learning, pp. 115–118.
  18. Kain, A., Macon, M.W., 1998. Spectral voice conversion for text-to-speech synthesis. In: Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing, pp. 285–288. · DOI · Paper page
  19. Kenny, O.P., Nelson, D.J., Bodenschatz, J.S., McMonagle, H.A., 1998. Separation of non-spontaneous and spontaneous speech. In: Proc. 1998 IEEE Internat. Conf. on Acoustics, Speech and Signal Processing, Vol. 1, pp. 573–576. · doi:10.1109/icassp.1998.674495
  20. Kounoudes, A., Naylor, P.A., Brookes, M., 2002. The DYPSA algorithm for estimation of glottal closure instants in voiced speech. In: Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing, Orlando, FL, USA, pp. I-349–I-352. · doi:10.1109/icassp.2002.5743726
  21. Kreiman, 1991, Comparing discrimination and recognition of unfamiliar voices, Speech Commun., 10, 265, 10.1016/0167-6393(91)90016-M · DOI
  22. Major, R.C., 2001. Foreign Accent: The Ontogeny and Phylogeny of Second Language Phonology. L. Erlbaum, Mahwah, NJ, ix, 209pp. · DOI · Paper page
  23. Makhoul, J., Berouti, M., 1979. High-frequency regeneration in speech coding systems. In: Berouti, M. (Ed.), Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing, pp. 428–431. · doi:10.1109/icassp.1979.1170672
  24. Martin, P., 2004. WinPitch LTL II, a Multimodal Pronunciation Software. ISCA.
  25. Matsumoto, 1973, Multidimensional representation of personal quality of vowels and its acoustical correlates, IEEE Trans. Audio Electroacoust., 21, 428, 10.1109/TAU.1973.1162507 · doi:10.1109/tau.1973.1162507
  26. McAllister, R., 1998. Second language perception and the concept of foreign accent. In: Proc. ESCA Workshop on Speech Technology in Language Learning (STiLL 98), Stockholm, Sweden, pp. 155–158.
  27. Menzel, W., Herron, D., Bonaventura, P., Morton, R., 2000. Automatic detection and correction of non-native English pronunciations, InSTILL, Dundee, Scotland, pp. 49–56.
  28. Moulines, 1990, Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones, Speech Commun., 9, 453, 10.1016/0167-6393(90)90021-Z · DOI · Paper page
  29. Moulines, 1995, Non-parametric techniques for pitch-scale and time-scale modification of speech, Speech Commun., 16, 175, 10.1016/0167-6393(94)00054-E · DOI · Paper page
  30. Munro, 1995, Non-segmental factors in foreign accent: ratings of filtered speech, Studies Second Language Acquisit., 17, 17, 10.1017/S0272263100013735 · DOI
  31. Munro, 1994, Evaluations of foreign accent in extemporaneous and read material, Language Testing, 11, 253, 10.1177/026553229401100302 · DOI
  32. Murray, 1999, Autonomy and language learning in a simulated environment, System, 27, 295, 10.1016/S0346-251X(99)00026-3 · DOI
  33. Nagano, K., Ozawa, K., 1990. English speech training using voice conversion. In: 1st Internat. Conf. on Spoken Language Processing (ICSLP 90), Kobe, Japan, pp. 1169–1172. · doi:10.21437/icslp.1990-309
  34. Neri, 2002, The pedagogy–technology interface in computer assisted pronunciation training, Comput. Assisted Language Learn., 15, 441, 10.1076/call.15.5.441.13473 · DOI · Paper page
  35. Neri, A., Cucchiarini, C., Strik, H., 2003. Automatic speech recognition for second language learning: how and why it actually works. In: Proc. 15th Internat. Congress of Phonetic Sciences, Barcelona, Spain, pp. 1157–1160.
  36. Paul, 1981, The spectral envelope estimation vocoder, IEEE Trans. Acoust. Speech Signal Process., 29, 786, 10.1109/TASSP.1981.1163643 · DOI
  37. Peabody, 2006, Towards automatic tone correction in non-native mandarin, Chinese Spoken Language Process., 602, 10.1007/11939993_62 · DOI
  38. Pennington, 1999, Computer-aided pronunciation pedagogy: promise, limitations, directions, Comput. Assisted Language Learn., 12, 427, 10.1076/call.12.5.427.5693 · DOI
  39. Probst, 2002, Enhancing foreign language tutors – in search of the golden speaker, Speech Commun., 37, 161, 10.1016/S0167-6393(01)00009-7 · DOI
  40. Repp, 1987, Categorical tendencies in imitating self-produced isolated vowels, Speech Commun., 6, 1, 10.1016/0167-6393(87)90065-3 · DOI
  41. Rogers, 1996, Prediction of foreign-accented speech intelligibility from segmental contrast measures, J. Acoust. Soc. Amer., 100, 2725, 10.1121/1.416179 · doi:10.1121/1.416179
  42. Ryan, 1975, Evaluative reactions of adolescents toward speakers of standard English and Mexican American accented English, J. Personality Social Psychol., 31, 855, 10.1037/h0076704 · DOI · Paper page
  43. Sambur, 1975, Selection of acoustic features for speaker identification, IEEE Trans. Acoust. Speech Signal Process., 23, 176, 10.1109/TASSP.1975.1162664 · doi:10.1109/tassp.1975.1162664
  44. Schairer, 1992, Native speaker reaction to non-native speech, Mod. Language J., 76, 309
  45. Sheffert, 2002, Learning to recognize talkers from natural, sinewave, and reversed speech samples, J. Exp. Psychol. Hum. Percept. Perform., 28, 1447, 10.1037/0096-1523.28.6.1447 · DOI
  46. Sundermann, D., Ney, H., Hoge, H., 2003. VTLN-based cross-language voice conversion. In: Proc. IEEE Workshop on Automatic Speech Recognition and Understanding. St. Thomas, US Virgin Islands, pp. 676–681. · doi:10.1109/asru.2003.1318521
  47. Sundström, A., 1998. Automatic prosody modification as a means for foreign language pronunciation training. In: Proc. ISCA Workshop on Speech Technology in Language Learning (STILL 98), Marholmen, Sweden, pp. 49–52.
  48. Tang, M., Wang, C., Seneff, S., 2001. Voice Transformations: From Speech Synthesis to Mammalian Vocalizations, Eurospeech 2001, Aalborg, Denmark. · doi:10.21437/eurospeech.2001-109
  49. Traunmüller, 1994, Conventional, biological and environmental factors in speech communication: a modulation theory, Phonetica, 51, 170, 10.1159/000261968 · DOI
  50. Turk, 2006, Robust processing techniques for voice conversion, Comput. Speech Language, 20, 441, 10.1016/j.csl.2005.06.001 · DOI
  51. van Lancker, 1985, Familiar voice recognition: pattern and parameters. Part I: recognition of backward voices, J. Phonetics, 13, 19, 10.1016/S0095-4470(19)30723-5 · DOI
  52. Vieru-Dimulescu, B., Mareüil, P.B.d., 2005. Contribution of prosody to the perception of a foreign accent:a study based on Spanish/Italian modified speech. In: Proc. ISCA Workshop on Plasticity in Speech Perception, London, UK, pp. 66–68.
  53. Wachowicz, 1999, Software that listens: it’s not a question of whether, it’s a question of how, CALICO J., 16, 253, 10.1558/cj.v16i3.253-276 · DOI
  54. Watson, 1989, Advances in computer-based speech training: Aids for the profoundly hearing impaired, Volta-Review, 91, 29
  55. Yan, Q., Vaseghi, S., Rentzos, D., Ho, C.-H., 2004. Analysis by synthesis of acoustic correlates of British, Australian and American accents, In: Proc. IEEE Internat. Conf. on Acoustics, Speech, and Signal Processing, 2004 (ICASSP’04), Montreal, Quebec, Canada, pp. I-637–I-640.
  56. Young, S.J., 1993. The HTK Hidden Markov Model Toolkit: Design and Philosophy. Department of Engineering, Cambridge University, UK, Tech. Rep. TR, 153.

Cited by

See all 130

Metadata

Authors
Felps, D., Bortfeld, H., & Gutierrez-Osuna, R.
Venue
Speech Communication
Publisher
Elsevier BV
Published
2009-10
Volume
51
Issue
10
Pages
920-932
ISSN
0167-6393 (ISSN-L)
OpenAlex
W2017742648
Cited by
130
References
6
FWCI
3.47
Type
journal article
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.