“Automatic spontaneous speech grading: a novel feature derivation technique using the crowd,”
Open accessAbstract
Vinay Shashidhar, Nishant Pandey, Varun Aggarwal. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI-based assessment tools have been used for spontaneous speech evaluation where the student's response is not known a priori.
“or the more complex problem of spontaneous speech evaluation where the student's response is not known apriori (Shashidhar et al., 2015)”
From Proactive and reactive engagement of artificial intelligence methods for education: a review (2023) · cites this paper as Shashidhar et al., 2015
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Towards an integrated crowdsourcing definition
Estellés-Arolas, E., & González-Ladrón-de-Guevara, F. (2012) · Journal of Information Science · doi:10.1177/0165551512437638 · Cited by 1,859 · see citing works
DARPA TIMIT:: acoustic-phonetic continuous speech corpus CD-ROM, NIST speech disc 1-1.1
Garofolo, J. S., Lamel, L., Fisher, W. M., Fiscus, J. G., Pallett, D., & Dahlgren, N. L. (1993) · doi:10.6028/nist.ir.4930 · Cited by 1,533 · see citing works
Fiscus, J. (2002) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.1997.659110 · Cited by 1,102 · see citing works
Crowdsourcing and Its Application in Marketing Activities
Whitla, P. (2009) · Contemporary Management Research · doi:10.7903/cmr.1145 · Cited by 342 · see citing works
Crowdsourcing Translation: Professional Quality from Non-Professionals
Zaidan, O. F., & Callison-Burch, C. (2011) · Cited by 312 · see citing works
Bernstein, M. S., Brandt, J., Miller, R. C., & Karger, D. R. (2011) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2047196.2047201 · Cited by 302 · see citing works
Cucchiarini, C., Strik, H., & Boves, L. (2000) · The Journal of the Acoustical Society of America · doi:10.1121/1.428279 · Cited by 293 · see citing works
Automatic scoring of non-native spontaneous speech in tests of spoken English
Zechner, K., Higgins, D., Xi, X., & Williamson, D. M. (2009) · Speech Communication · doi:10.1016/j.specom.2009.04.009 · Cited by 250 · see citing works
Little, D. (2007) · Modern Language Journal · doi:10.1111/j.1540-4781.2007.00627_2.x · Cited by 206 · see citing works
The problem of logical omniscience, I
STALNAKER, R. C. (1991) · Synthese · doi:10.1007/bf00413506 · Cited by 185 · see citing works
An Evaluation of Aggregation Techniques in Crowdsourcing
Nguyen, Q. V. H., Nguyên, T. T., Tran, L. N., & Aberer, K. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41154-0_1 · Cited by 181 · see citing works
Little, D. (2006) · Language Teaching · doi:10.1017/s0261444806003557 · Cited by 147 · see citing works
Stumping e-rater:challenging the validity of automated essay scoring
Powers, D. E., Burstein, J. C., Chodorow, M. S., Fowles, M. E., & Kukich, K. (2002) · Computers in Human Behavior · doi:10.1016/s0747-5632(01)00052-8 · Cited by 146 · see citing works
Automatic text-independent pronunciation scoring of foreign language student speech
Neumeyer, L., Franco, H., Weintraub, M., & Price, P. J. (2002) · International Conference on Spoken Language Processing (ICSLP) · doi:10.1109/icslp.1996.607890 · Cited by 143 · see citing works
Lejk, M., & Wyvill, M. (2001) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930120093887 · Cited by 126 · see citing works
(1995) · Medical Entomology and Zoology · Cited by 123 · see citing works
Age at Migration, Language Proficiency, and Socioeconomic Outcomes: Evidence From Australia
Guven, C., & Islam, A. (2015) · Demography · doi:10.1007/s13524-015-0373-6 · Cited by 121 · see citing works
An HMM-based system for automatic segmentation and alignment of speech
Sjölander, K. (2003) · Cited by 99 · see citing works
The SRI EduSpeak System: Recognition and Pronunciation Scoring for Language Learning
Franco, H., Abrash, V., Precoda, K., Bratt, H., Rao, R., Butzberger, J., Rossier, R., & Cesari, F. (2007) · Cited by 67 · see citing works
On Aggregating Labels from Multiple Crowd Workers to Infer Relevance of Documents
Hosseini, M., Cox, I. J., Milić-Frayling, N., et al. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-28997-2_16 · Cited by 66 · see citing works
Warping time for more effective real-time crowdsourcing
Lasecki, W. S., Miller, C. D., & Bigham, J. P. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466269 · Cited by 65 · see citing works
Automatic evaluation of Dutch pronunciation by using speech recognition technology
Cucchiarini, C., Strik, H., & Boves, L. (2002) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.1997.659144 · Cited by 52 · see citing works
Assessing Oral Proficiency: A Comparison of Holistic and Objective Measures
Halleck, G. B. (1995) · Modern Language Journal · doi:10.1111/j.1540-4781.1995.tb05434.x · Cited by 48 · see citing works
Assessing Crowdsourcing Quality through Objective Tasks
Aker, A., El‐Haj, M., Albakour, M., et al. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/55mecrh5qage · Cited by 47 · see citing works
They Can Help: Using Crowdsourcing to Improve the Evaluation of Grammatical Error Detection Systems
Madnani, N., Chodorow, M. S., Tetreault, J., et al. (2011) · Cited by 35 · see citing works
Rethinking Grammatical Error Annotation and Evaluation with the Amazon Mechanical Turk
Tetreault, J., Filatova, E., & Chodorow, M. S. (2010) · Cited by 28 · see citing works
Methods for pronunciation assessment in computer aided language learning
Peabody, M. (2011) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 22 · see citing works
The Wisdom of the Crowd’s Ear: Speech Accent Rating and Annotation with Amazon Mechanical Turk
Kunath, S. A., & Weinberger, S. H. (2010) · North American Chapter of the Association for Computational Linguistics · Cited by 22 · see citing works
Automatic assessment of pronunciation quality
Dong, B., Zhao, Q., Zhang, J., et al. (2005) · doi:10.1109/chinsl.2004.1409605 · Cited by 12 · see citing works
Li, H., Wang, S., Liang, J., et al. (2009) · Interspeech (USB) · doi:10.21437/interspeech.2009-553 · Cited by 9 · see citing works
Phonological modeling of mispronunciation gradations in L2 English speech of L1 Chinese learners
Wang, H., Qian, X., & Meng, H. M. L. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6855101 · Cited by 9 · see citing works
Wang, H., & Meng, H. M. L. (2012) · doi:10.1109/icsda.2012.6422468 · Cited by 6 · see citing works
Shallow Analysis Based Assessment of Syntactic Complexity for Automated Speech Scoring
Bhat, S., Xue, H., & Yoon, S. (2014) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/v1/p14-1123 · Cited by 6 · see citing works
Perceptual differentiation modeling explains phoneme mispronunciation by non-native speakers
Koniaris, C., & Engwall, O. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947655 · Cited by 5 · see citing works
Can the Internet grade math? Crowdsourcing a complex scoring task and picking the optimal crowd size
Van Houdnos, N. (2018) · doi:10.1184/r1/6586484 · Cited by 4 · see citing works
Creation of Speaking Tests for EFL Communication Classes
Hagley, E. (2010) · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 10
- Christopher Brumfit and Christopher J Brumfit. 1984. Communicative methodology in language teaching: The roles of fluency and accuracy, volume 129. Cambridge University Press Cambridge.
- Eric John Dobson. 1957. English Pronunciation, 1500-1700: Phonology, volume 2. Clarendon Press.
- Cambridge EOCL Examinations. 2011. Using the CEFR: Principles of good practice. at University of Cambridge.
- Jeff Howe. 2006. The rise of crowdsourcing. Wired magazine, 14(6):1-4.
- Akinori Ito, Tadao Nagasawa, Hirokazu Ogasawara, Motoyuki Suzuki, and Shozo Makino. 2006. Au- tomatic detection of english mispronunciation using speaker adaptation and automatic assessment of en- glish intonation and rhythm. Educational technol- ogy research, 29(1):13-23.
- Gabriele Paolacci, Jesse Chandler, and Panagiotis G Ipeirotis. 2010. Running experiments on amazon mechanical turk. Judgment and Decision making, 5(5):411-419.
- Mitchell Aaron Peabody. 2011. Methods for pronunci- ation assessment in computer aided language learn- ing. Ph.D. thesis, Massachusetts Institute of Tech- nology.
- Robert Stalnaker. 1999. The problem of logical omni- science, ii. context and content: Essays on intention- ality in speech and thought (pp. 255-273).
- Steve Young, Gunnar Evermann, Mark Gales, Thomas Hain, Dan Kershaw, Xunying Liu, Gareth Moore, Julian Odell, Dave Ollason, Dan Povey, et al. 2006. The htk book (for htk version 3.4). Cambridge uni- versity engineering department, 2(2):2-3.
- Klaus Zechner, Derrick Higgins, and Xiaoming Xi. 2007. Speechrater: A construct-driven approach to scoring spontaneous non-native speech. Proc. SLaTE.
Cited by
Sorted by citations
Proactive and reactive engagement of artificial intelligence methods for education: a review
Mallik, S., & Gangopadhyay, A. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1151391 · Cited by 139 · see citing works
- Authors
- Shashidhar, V., Pandey, N., & Aggarwal, V.
- ORCID
- Aggarwal, Varun
- Venue
- Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2015
- Pages
- 1085-1094
- OpenAlex
- W2250604663
- Cited by
- 24
- References
- 36
- Citations by year
- 2024: 1 · 2023: 1 · 2022: 18 · 2019: 1 · 2018: 1 · 2016: 2
- FWCI
- 0.56
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- spontaneous speech, crowdsourcing, automated assessment, speech quality assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
