Foreign accent conversion in computer assisted pronunciation training
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Learners of a second language practice their pronunciation by listening to and imitating utterances from native speakers. Recent research has shown that choosing a well-matched native speaker to imitate can have a positive impact on pronunciation training. Here we propose a voice-transformation technique that can be used to generate the (arguably) ideal voice to imitate: the own voice of the learner with a native accent. Our work extends previous research, which suggests that providing learners with prosodically corrected versions of their utterances can be a suitable form of feedback in computer assisted pronunciation training. Our technique provides a conversion of both prosodic and segmental characteristics by means of a pitch-synchronous decomposition of speech into glottal excitation and spectral envelope. We apply the technique to a corpus containing parallel recordings of foreign-accented and native-accented utterances, and validate the resulting accent conversions through a series of perceptual experiments. Our results indicate that the technique can reduce foreign accentedness without significantly altering the voice quality properties of the foreign speaker. Finally, we propose a pedagogical strategy for integrating accent conversion as a form of behavioral shaping in computer assisted pronunciation training.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A note on two problems in connexion with graphs
Dijkstra, E. W. (1959) · Numerische Mathematik · doi:10.1007/bf01386390 · Cited by 29,065
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585
The Biological Foundations of Language
Lenneberg, E. H. (1967) · Hospital Practice · doi:10.1080/21548331.1967.11707799 · Cited by 6,605
(1960) · Medical Journal of Australia · doi:10.5694/j.1326-5377.1960.tb76238.x · Cited by 2,178
Foreign Accent, Comprehensibility, and Intelligibility in the Speech of Second Language Learners
Munro, M. J., & Derwing, T. M. (1995) · Language Learning · doi:10.1111/j.1467-1770.1995.tb00963.x · Cited by 1,545
Lyster, R. (2001) · Language Learning · doi:10.1111/j.1467-1770.2001.tb00019.x · Cited by 172
Research works printed but not linked: 56
- Abe, M., Nakamura, S., Shikano, K., Kuwabara, H., 1988. Voice conversion through vector quantization. In: Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing, New York, NY, pp. 655–658. · DOI · Paper page
- Anisfeld, 1962, Evaluational reactions to accented English speech, J. Abnorm. Social Psychol., 65, 223, 10.1037/h0045060 · DOI
- Arslan, 1997, A study of temporal features and frequency characteristics in American English foreign accent, J. Acoust. Soc. Amer., 102, 28, 10.1121/1.419608 · DOI
- Arslan, L.M., Talkin, D., 1997. Voice Conversion By Codebook Mapping Of Line Spectral Frequencies And Excitation Spectrum, Eurospeech’97, Rhodes, Greece, pp. 1347–1350. · DOI
- Arthur, 1974, Evaluation reactions of college students to dialect differences in the English of Mexican–Americans, Language Speech, 17, 255, 10.1177/002383097401700304 · doi:10.1177/002383097401700304
- Childers, 1989, Voice conversion, Speech Commun., 8, 147, 10.1016/0167-6393(89)90041-1 · DOI
- Chun, 1998, Signal analysis software for teaching discourse intonation, Language Learn. Technol., 2, 61, 10.64152/10125/25033 · DOI
- Compton, 1963, Effects of filtering and vocal duration upon identification of speakers, aurally, J. Acoust. Soc. Amer., 35, 1748, 10.1121/1.1918810 · DOI
- Derwing, 1998, Evidence in favor of a broad framework for pronunciation instruction, Language Learn., 48, 393, 10.1111/0023-8333.00047 · DOI · Paper page
- Derwing, 1998, Evidence in favour of a broad framework for pronunciation instruction, Language Learn., 48, 393, 10.1111/0023-8333.00047 · DOI · Paper page
- Eskenazi, 1999, Using automatic speech processing for foreign language pronunciation tutoring: some issues and a prototype, Language Learn. Technol., 2, 62, 10.64152/10125/25043 · DOI · Paper page
- Eskenazi, M., Hansma, S., 1998. The fluency pronunciation trainer. In: Proc. STiLL Workshop on Speech Technology in Language Learning. · doi:10.21437/eurospeech.1999-219
- Griffin, 1984, Signal estimation from modified short-time Fourier transform, IEEE Trans. Acoust. Speech Signal Process., 32, 236, 10.1109/TASSP.1984.1164317 · DOI · Paper page
- Hansen, T.K., 2006. Computer assisted pronunciation training: the four ‘K’s of feedback. In: 4th Internat. Conf. on Multimedia and Information and Communication Technologies in Education, Seville, Spain, pp. 342–346.
- Hincks, 2003, Speech technologies for pronunciation feedback and evaluation, ReCALL, 15, 3, 10.1017/S0958344003000211 · DOI
- Huckvale, M., Yanagisawa, K., 2007. Spoken language conversion with accent morphing. In: Proc. ISCA Speech Synthesis Workshop, Bonn, Germany, pp. 64–70.
- Jilka, M., Möhler, G., 1998. Intonational foreign accent: speech technology and foreign language teaching. In: Proc. ESCA Workshop on Speech Technology in Language Learning, pp. 115–118.
- Kain, A., Macon, M.W., 1998. Spectral voice conversion for text-to-speech synthesis. In: Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing, pp. 285–288. · DOI · Paper page
- Kenny, O.P., Nelson, D.J., Bodenschatz, J.S., McMonagle, H.A., 1998. Separation of non-spontaneous and spontaneous speech. In: Proc. 1998 IEEE Internat. Conf. on Acoustics, Speech and Signal Processing, Vol. 1, pp. 573–576. · doi:10.1109/icassp.1998.674495
- Kounoudes, A., Naylor, P.A., Brookes, M., 2002. The DYPSA algorithm for estimation of glottal closure instants in voiced speech. In: Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing, Orlando, FL, USA, pp. I-349–I-352. · doi:10.1109/icassp.2002.5743726
- Kreiman, 1991, Comparing discrimination and recognition of unfamiliar voices, Speech Commun., 10, 265, 10.1016/0167-6393(91)90016-M · DOI
- Major, R.C., 2001. Foreign Accent: The Ontogeny and Phylogeny of Second Language Phonology. L. Erlbaum, Mahwah, NJ, ix, 209pp. · DOI · Paper page
- Makhoul, J., Berouti, M., 1979. High-frequency regeneration in speech coding systems. In: Berouti, M. (Ed.), Proc. Internat. Conf. on Acoustics, Speech, and Signal Processing, pp. 428–431. · doi:10.1109/icassp.1979.1170672
- Martin, P., 2004. WinPitch LTL II, a Multimodal Pronunciation Software. ISCA.
- Matsumoto, 1973, Multidimensional representation of personal quality of vowels and its acoustical correlates, IEEE Trans. Audio Electroacoust., 21, 428, 10.1109/TAU.1973.1162507 · doi:10.1109/tau.1973.1162507
- McAllister, R., 1998. Second language perception and the concept of foreign accent. In: Proc. ESCA Workshop on Speech Technology in Language Learning (STiLL 98), Stockholm, Sweden, pp. 155–158.
- Menzel, W., Herron, D., Bonaventura, P., Morton, R., 2000. Automatic detection and correction of non-native English pronunciations, InSTILL, Dundee, Scotland, pp. 49–56.
- Moulines, 1990, Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones, Speech Commun., 9, 453, 10.1016/0167-6393(90)90021-Z · DOI · Paper page
- Moulines, 1995, Non-parametric techniques for pitch-scale and time-scale modification of speech, Speech Commun., 16, 175, 10.1016/0167-6393(94)00054-E · DOI · Paper page
- Munro, 1995, Non-segmental factors in foreign accent: ratings of filtered speech, Studies Second Language Acquisit., 17, 17, 10.1017/S0272263100013735 · DOI
- Munro, 1994, Evaluations of foreign accent in extemporaneous and read material, Language Testing, 11, 253, 10.1177/026553229401100302 · DOI
- Murray, 1999, Autonomy and language learning in a simulated environment, System, 27, 295, 10.1016/S0346-251X(99)00026-3 · DOI
- Nagano, K., Ozawa, K., 1990. English speech training using voice conversion. In: 1st Internat. Conf. on Spoken Language Processing (ICSLP 90), Kobe, Japan, pp. 1169–1172. · doi:10.21437/icslp.1990-309
- Neri, 2002, The pedagogy–technology interface in computer assisted pronunciation training, Comput. Assisted Language Learn., 15, 441, 10.1076/call.15.5.441.13473 · DOI · Paper page
- Neri, A., Cucchiarini, C., Strik, H., 2003. Automatic speech recognition for second language learning: how and why it actually works. In: Proc. 15th Internat. Congress of Phonetic Sciences, Barcelona, Spain, pp. 1157–1160.
- Paul, 1981, The spectral envelope estimation vocoder, IEEE Trans. Acoust. Speech Signal Process., 29, 786, 10.1109/TASSP.1981.1163643 · DOI
- Peabody, 2006, Towards automatic tone correction in non-native mandarin, Chinese Spoken Language Process., 602, 10.1007/11939993_62 · DOI
- Pennington, 1999, Computer-aided pronunciation pedagogy: promise, limitations, directions, Comput. Assisted Language Learn., 12, 427, 10.1076/call.12.5.427.5693 · DOI
- Probst, 2002, Enhancing foreign language tutors – in search of the golden speaker, Speech Commun., 37, 161, 10.1016/S0167-6393(01)00009-7 · DOI
- Repp, 1987, Categorical tendencies in imitating self-produced isolated vowels, Speech Commun., 6, 1, 10.1016/0167-6393(87)90065-3 · DOI
- Rogers, 1996, Prediction of foreign-accented speech intelligibility from segmental contrast measures, J. Acoust. Soc. Amer., 100, 2725, 10.1121/1.416179 · doi:10.1121/1.416179
- Ryan, 1975, Evaluative reactions of adolescents toward speakers of standard English and Mexican American accented English, J. Personality Social Psychol., 31, 855, 10.1037/h0076704 · DOI · Paper page
- Sambur, 1975, Selection of acoustic features for speaker identification, IEEE Trans. Acoust. Speech Signal Process., 23, 176, 10.1109/TASSP.1975.1162664 · doi:10.1109/tassp.1975.1162664
- Schairer, 1992, Native speaker reaction to non-native speech, Mod. Language J., 76, 309
- Sheffert, 2002, Learning to recognize talkers from natural, sinewave, and reversed speech samples, J. Exp. Psychol. Hum. Percept. Perform., 28, 1447, 10.1037/0096-1523.28.6.1447 · DOI
- Sundermann, D., Ney, H., Hoge, H., 2003. VTLN-based cross-language voice conversion. In: Proc. IEEE Workshop on Automatic Speech Recognition and Understanding. St. Thomas, US Virgin Islands, pp. 676–681. · doi:10.1109/asru.2003.1318521
- Sundström, A., 1998. Automatic prosody modification as a means for foreign language pronunciation training. In: Proc. ISCA Workshop on Speech Technology in Language Learning (STILL 98), Marholmen, Sweden, pp. 49–52.
- Tang, M., Wang, C., Seneff, S., 2001. Voice Transformations: From Speech Synthesis to Mammalian Vocalizations, Eurospeech 2001, Aalborg, Denmark. · doi:10.21437/eurospeech.2001-109
- Traunmüller, 1994, Conventional, biological and environmental factors in speech communication: a modulation theory, Phonetica, 51, 170, 10.1159/000261968 · DOI
- Turk, 2006, Robust processing techniques for voice conversion, Comput. Speech Language, 20, 441, 10.1016/j.csl.2005.06.001 · DOI
- van Lancker, 1985, Familiar voice recognition: pattern and parameters. Part I: recognition of backward voices, J. Phonetics, 13, 19, 10.1016/S0095-4470(19)30723-5 · DOI
- Vieru-Dimulescu, B., Mareüil, P.B.d., 2005. Contribution of prosody to the perception of a foreign accent:a study based on Spanish/Italian modified speech. In: Proc. ISCA Workshop on Plasticity in Speech Perception, London, UK, pp. 66–68.
- Wachowicz, 1999, Software that listens: it’s not a question of whether, it’s a question of how, CALICO J., 16, 253, 10.1558/cj.v16i3.253-276 · DOI
- Watson, 1989, Advances in computer-based speech training: Aids for the profoundly hearing impaired, Volta-Review, 91, 29
- Yan, Q., Vaseghi, S., Rentzos, D., Ho, C.-H., 2004. Analysis by synthesis of acoustic correlates of British, Australian and American accents, In: Proc. IEEE Internat. Conf. on Acoustics, Speech, and Signal Processing, 2004 (ICASSP’04), Montreal, Quebec, Canada, pp. I-637–I-640.
- Young, S.J., 1993. The HTK Hidden Markov Model Toolkit: Design and Philosophy. Department of Engineering, Cambridge University, UK, Tech. Rep. TR, 153.
Cited by
The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods
Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., Kinnunen, T., & Ling, Z. (2018) · The Speaker and Language Recognition Workshop (Odyssey 2018) · doi:10.21437/odyssey.2018-28 · Cited by 305
The Voice Conversion Challenge 2016
Toda, T., Chen, L.-H., Saito, D., Villavicencio, F., Wester, M., Wu, Z., & Yamagishi, J. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-1066 · Cited by 156
Voice Conversion Challenge 2020 –- Intra-lingual semi-parallel and cross-lingual voice conversion –-
Yi, Z., Huang, W.-C., Tian, X., Yamagishi, J., Das, R. K., Kinnunen, T., Ling, Z.-H., & Toda, T. (2020) · Joint Workshop for the Blizzard Challenge and Voice Conversion Challenge 2020 · doi:10.21437/vcc_bc.2020-14 · Cited by 126
Khanjani, Z., Watson, G., & Janeja, V. P. (2023) · Frontiers in Big Data · doi:10.3389/fdata.2022.1001063 · Cited by 121
Metadata
- Authors
- Felps, D., Bortfeld, H., & Gutierrez-Osuna, R.
- Venue
- Speech Communication
- Publisher
- Elsevier BV
- Published
- 2009-10
- Volume
- 51
- Issue
- 10
- Pages
- 920-932
- ISSN
- 0167-6393 (ISSN-L)
- OpenAlex
- W2017742648
- Cited by
- 130
- References
- 6
- FWCI
- 3.47
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.