CoChat
Sign Up
Cited by 10Language Development and DisordersReading and Literacy DevelopmentSpeech Recognition and Synthesis

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Investigating the potential of NLP-driven linguistic and acoustic features for predicting human scores of children’s oral language proficiency

Hunte, Melissa R., McCormick, Samantha, Shah, Maitree, Lau, Clarissa, Jang, Eunice Eunhee·2021·Assessment in Education: Principles, Policy & Practice
DOI 10.1080/0969594x.2021.1999209
Open this paper in CoChat — free account

Abstract

Children’s oral language proficiency (OLP) is integral for developing literacy skills. Storytelling or retelling is often used by parents and educators to elicit children’s OLP, yet it is less commonly used for assessment purposes. Leveraged by natural language processing and machine learning, this study examined the extent to which computational linguistic and acoustic indices predict human ratings of children’s (n=184 aged 9 to 11) OLP using two story retell stimuli presented in written and aural forms. Human raters scored children’s OLP on five oral proficiency criteria: vocabulary, grammar, idea development, task-fulfilment, and speech delivery, using a 4-point scale, and linguistic and acoustic features were used to predict each criterion. Results showed the efficacy of automated indices to predict human scores of children’s OLP. This study calls for attention to discrepancies in human and machine speech delivery scores and stimulus effects on story retelling performance among children of different language backgrounds.

Overview

Integrity signals

Registry

Crossref · 10.1080/0969594x.2021.1999209

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 2 references

Research works printed but not linked: 105

  1. Akaike H., 1973, Second international symposium on information theory, 267
  2. Albrecht, S., Cullen, C., Davies, K., Dunlop, M., Elliott, M. & Stevenson, L. (2018). Cambridge Assessment English. Cambridge University Press. https://www.cambridgeenglish.org/Images/461823-young-learners-revision-publication.pdf
  3. Alim S. A., 2018, From natural to artificial intelligence: Algorithms and applications, 2
  4. Bachman L. F., 2010, Language assessment in practice: Developing language assessments and justifying their use in the real world
  5. Barzilay R., 2005, Proceedings of the 43rd annual meeting of the association for computational linguistics, 141
  6. 10.1162/coli.2008.34.1.1 · DOI
  7. 10.1177/0265532210364404 · DOI
  8. Bishop C. M, 2006, Pattern recognition and machine learning
  9. Bolaños D., 2013, Journal of Educational Psychology, 105, 1142, 10.1037/a0031479 · DOI
  10. Burstein J, 2009, Lecture notes in computer science: Vol. 5449. Computational linguistics and intelligent text processing, 6
  11. Cain K., 2007, Children’s comprehension problems in oral and written language: A cognitive perspective, 283
  12. Cambridge Assessment English (CAE). (2021). Pre a1 starters, a1 movers and a2 flyers handbook for teachers. https://www.cambridgeenglish.org/Images/357180-starters-movers-and-flyers-handbook-for-teachers-2021.pdf
  13. Campbell J.P., 1997, Proceedings of the IEEE, 85, 1437, 10.1109/5.628714 · DOI
  14. 10.1044/0161-1461(2001/004) · DOI
  15. 10.1177/0265532210364405 · DOI
  16. Chen H., 2013, Proceedings of the 2013 conference on empirical methods in natural language processing, 1741
  17. Chen, L., Zechner, K., Yoon, S. Y., Evanini, K., Wang, X., Loukina, A., Tao, J., Davis, L., Lee, C. M., Mundkowsky, R., Lu, C., Leong, C. W. & Gyawali, B. (2018). Automated scoring of nonnative speech using the SpeechRaterSM v. 5.0 engine. ETS Research Report Series, 1, 1-31: Educational Testing Service. https://doi.org/10.1002/ets2.12198 · DOI
  18. Chodorow, M. & Burstein, J. (2004). Beyond essay length: Evaluating e-rater’s performance on TOEFL essays. TOEFL Research Report Series, 1, i-83.Educational Testing Service . https://doi.org/10.1002/j.2333-8504.2004.tb01931.x · DOI
  19. Collins-Thompson K., 2004, Proceedings of the human language technology conference of the North American chapter of the association for computational linguistics, 193
  20. Cordeiro J., 2004, Proceedings of 4th international workshop on pattern recognition in information systems, 128
  21. 10.1080/09296171003643098 · DOI
  22. 10.1002/9781444302776 · DOI
  23. 10.1121/1.1471894 · DOI
  24. Cumbal R., 2021, Proceedings of Interspeech 2021, 4463, 10.21437/Interspeech.2021-2140 · DOI · Paper page
  25. Döllinger M., 2012, The Laryngoscope, 122, 2511, 10.1002/lary.23568 · DOI
  26. Dong G., 2018, Feature engineering for machine learning and data analytics, 1
  27. Dong L., 2011, Proceedings of the17th international congress of phonetic sciences meeting
  28. Evanini K., 2019, Automated speaking assessment: Using language technologies to score spontaneous speech, 3, 10.4324/9781315165103-1 · DOI
  29. Eyben F., 2013, Proceedings of the 21st ACM international conference on multimedia, 835, 10.1145/2502081.2502224 · DOI
  30. Farrús M., 2007, Proceedings of the 8th annual conference of the international speech communication association, 778
  31. 10.1044/1058-0360(2013/12-0083) · DOI
  32. Fielding L., 2007, Annual growth for all students: Catch-up growth for those who are behind
  33. Fillmore L. W., 2018, What teachers need to know about language, 8
  34. Firth J. R, 1957, Papers in linguistics, 1934–1951
  35. 10.1037/a0015825 · DOI
  36. 10.1207/S1532799XSSR0503_3 · DOI
  37. Gerosa M., 2009, Proceedings of the 2nd workshop on child, computer and interaction, 1
  38. 10.1891/9780826123497 · DOI
  39. 10.1044/jshr.3506.1303 · DOI
  40. 10.1080/2331186X.2021.1935647 · DOI
  41. Gupta D., 2018, Advances in intelligent systems and computing, Vol 664. Speech and language processing for human-machine communications, 195
  42. Hacki T., 1999, International Journal of Pediatric Otorhinolaryngology, 49, S141, 10.1016/S0165-5876(99)00150-0 · DOI
  43. Hagen A., 2007, Speech Communication, 49, 861, 10.1016/j.specom.2007.05.004 · DOI
  44. Han J., 2011, Data mining: Concepts and techniques
  45. Hannah L., 2021, Investigating the effects of task type and linguistic background on accuracy in automated speech recognition systems: Implications for use in language assessment of young learners
  46. Hasselgreen A., 2016, Language Testing, 22, 337, 10.1191/0265532205lt312oa · DOI
  47. 10.1044/1058-0360(2009/08-0024) · DOI
  48. 10.1177/0265532217734240 · DOI
  49. Huber J. E., 1999, The Journal of the Acoustical Society of America, 106, 1532, 10.1121/1.427150 · DOI
  50. Hymes D. H, 1972, Sociolinguistics. Selected readings, 269
  51. Juel C., 2003, Educational Leadership, 60, 12
  52. Kang O., 2017, Assessment in second language pronunciation, 10.4324/9781315170756 · DOI
  53. 10.1080/15434303.2018.1451531 · DOI
  54. Kohavi R., 1998, The springer international series in engineering and computer science, Vol 453. Feature extraction, construction and selection, 33, 10.1007/978-1-4615-5725-8_3 · DOI
  55. Kusner M., 2015, Proceedings of the 32nd international conference on machine learning, 957
  56. Landauer T. K., 2002, Automated essay scoring: A cross-disciplinary perspective, 305
  57. 10.1121/1.426686 · DOI
  58. 10.1159/000485974 · DOI
  59. Lin C. Y., 2004, Preceedings of the 42nd Annual Meeting of the Association for Computational Linguistics
  60. 10.3102/10769986022003249 · DOI
  61. Lord F. M., 1980, Applications of item response theory to practical testing problems
  62. 10.1017/CBO9780511733017 · DOI
  63. Malec A., 2017, Canadian Journal of Education/Revue Canadienne De L’éducation, 40, 362
  64. 10.1111/j.1467-8624.1980.tb02504.x · DOI
  65. 10.1017/CBO9780511809071 · DOI · Paper page
  66. McCarthy, P. M. (2005). An assessment of the range and usefulness of lexical diversity measures and the potential of the measure of textual, lexical diversity (MTLD) [Unpublished doctoral dissertation]. The University of Memphis.
  67. McKay P., 2006, Assessing young language learners
  68. 10.3758/s13428-012-0258-1 · DOI
  69. 10.1177/1525740117751897 · DOI
  70. Milton J, 2013, L2 vocabulary acquisition, knowledge and use: New perspectives on assessment and corpus analysis, 57
  71. Mladenic D., 1999, Proceedings of the sixteenth international conference on machine learning, 258
  72. Mohsen M. A., 2020, Journal of Psycholinguistic Research, 50, 543, 10.1007/s10936-020-09690-y · DOI
  73. Morris J., 1991, Computational Linguistics, 17, 21
  74. Napolitano D., 2015, Proceedings of the 2015 conference of the North American chapter of the association for computational linguistics: Demonstrations, 96
  75. Nippold M. A, 2016, Later language development: School-age children, adolescents, and young adults
  76. 10.1080/136820310000108133 · DOI
  77. Pearson Education. (2019). Versant English test: Test description and validation summary. https://www.pearson.com/content/dam/one-dot-com/one-dot-com/english/SupportingDocs/Versant/ValidationSummary/Versant-English-Test-Description-Validation-Report.pdf
  78. Pearson Education. (March, 2021). Global scale of English assessment framework for young learners. https://www.pearson.com/content/dam/one-dot-com/one-dot-com/english/SupportingDocs/GSE_Assessment_Young.pdf
  79. Perelman L, 2014, Assessing Writing, 21, 104, 10.1016/j.asw.2014.05.001 · DOI
  80. 10.1044/lle19.4.119 · DOI
  81. Peterson S. S, 2016, What Works? Research into Practice, 62, 1
  82. Quinlan, T., Higgins, D. & Wolff, S. (2009). Evaluating the construct‐coverage of the e‐rater® scoring engine. ETS Research Report Series, 1, 1-35: Educational Testing Service, https://doi.org/10.1002/j.2333-8504.2009.tb02158.x · DOI
  83. 10.1007/s40593-017-0143-2 · DOI
  84. 10.1027/1015-5759.23.1.39 · DOI
  85. Sahidullah M., 2010, International Journal of Biometrics, 2, 358, 10.1504/IJBM.2010.035450 · DOI
  86. Samejima F, 1969, Psychometrika Monograph Supplement, 34, 1
  87. 10.1037/0022-0663.96.1.119 · DOI
  88. 10.1214/aos/1176344136 · DOI · Paper page
  89. 10.1145/505282.505283 · DOI
  90. 10.1016/j.asw.2013.04.001 · DOI
  91. 10.1121/1.415421 · DOI
  92. So, Y., Wolf, M. K., Hauck, M. C., Mollaun, P., Rybinski, P., Tumposky, D. & Wang, J. (2015). TOEFL Junior® design framework. Educational Testing Service. ETS Research Report Series, TOEFL Junior Research Report No. 02. https://files.eric.ed.gov/fulltext/EJ1109688.pdf · DOI
  93. Stein N. L., 1979, New directions in discourse processing, 53
  94. 10.1016/j.protcy.2013.12.124 · DOI
  95. Uchikoshi Y., 2016, Literacy Research, 65, 236
  96. 10.1002/9781119422730 · DOI
  97. 10.1080/00220671.1985.10885604 · DOI · Paper page
  98. Witten I. H., 2011, Data mining: Practical machine learning tools and techniques
  99. 10.1145/507338.507355 · DOI
  100. 10.1207/S1532799XSSR0503_2 · DOI
  101. 10.1177/0265532211425673 · DOI
  102. Yoon S., 2018, Speech Communication, 99, 221, 10.1016/j.specom.2018.04.003 · DOI
  103. Zechner, K., Chen, L., Davis, L., Evanini, K., Lee, C. M., Leong, C. W., Wang, X. & Yoon, S. Y. (2015). Automated scoring of speaking tasks in the test of English‐for‐Teaching (TEFT™). ETS Research Report Series, 2, 1-17: Education Testing Services. https://doi.org/10.1002/ets2.12080 · DOI
  104. Zheng A., 2018, Feature engineering for machine learning: Principles and techniques for data scientists
  105. 10.1016/j.knosys.2017.01.006 · DOI

Cited by

Sorted by citations

See all 10

Metadata

Authors
Hunte, M. R., McCormick, S., Shah, M., Lau, C., & Jang, E. E.
Affiliations
University of Toronto (CA)
Corresponding author
Hunte, Melissa R.
ORCID
Venue
Assessment in Education: Principles, Policy & Practice
Publisher
Informa UK Limited
Published
2021-07-04
Volume
28
Issue
4
Pages
477-505
ISSN
  • 0969-594X (ISSN-L)
  • 1465-329X
DOI
10.1080/0969594x.2021.1999209
OpenAlex
W4205974171
Cited by
10
References
2
Citations by year
2026: 2 · 2025: 1 · 2024: 4 · 2023: 3
FWCI
2.57
Type
journal article
Language
en
Open access
closed
Locations
Assessment in Education Principles Policy and Practice (publishedVersion)
Topics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Language Development and Disorders
  • Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Keywords
retelling, automated grading, linguistic features, acoustic features, human-rater agreement, speech delivery, vocabulary, grammar, idea development, language background, literacy development

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Investigating the potential of NLP-driven linguistic and acoustic features for predicting human scores of children’s oral language proficiency. https://app.cochat.ai/papers/10.1080/0969594x.2021.1999209
The paper
Hunte, M. R., McCormick, S., Shah, M., Lau, C., & Jang, E. E. (2021). Investigating the potential of NLP-driven linguistic and acoustic features for predicting human scores of children’s oral language proficiency. Assessment in Education: Principles, Policy & Practice. https://doi.org/10.1080/0969594x.2021.1999209