The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Automatic detection of articulation disorders in children with cleft lip and palate
Abstract
Speech of children with cleft lip and palate (CLP) is sometimes still disordered even after adequate surgical and nonsurgical therapies. Such speech shows complex articulation disorders, which are usually assessed perceptually, consuming time and manpower. Hence, there is a need for an easy to apply and reliable automatic method. To create a reference for an automatic system, speech data of 58 children with CLP were assessed perceptually by experienced speech therapists for characteristic phonetic disorders at the phoneme level. The first part of the article aims to detect such characteristics by a semiautomatic procedure and the second to evaluate a fully automatic, thus simple, procedure. The methods are based on a combination of speech processing algorithms. The semiautomatic method achieves moderate to good agreement (κ≈0.6) for the detection of all phonetic disorders. On a speaker level, significant correlations between the perceptual evaluation and the automatic system of 0.89 are obtained. The fully automatic system yields a correlation on the speaker level of 0.81 to the perceptual evaluation. This correlation is in the range of the inter-rater correlation of the listeners. The automatic speech evaluation is able to detect phonetic disorders at an experts’level without any additional human postprocessing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Regarding the fully automatic evaluation, the accuracy percentage at the frame level ranges from 94.2% to 99.1%, with sensitivity from 52.6% to 71.1%. Moreover, at the phoneme level, the accuracy percentage ranged from 88.5% to 99.6%, with sensitivity from 56.9% to 76.9%. Lastly, at the word level, the accuracy percentage ranged from 68.6% to 98.2%, with sensitivity from 50.7% to 75.8%. (The study”
From The role of AI in the diagnosis of speech and language disorders: A systematic mapping study (2025) · cites this paper as [43]
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
(1992) · Cited by 23,431
Witten, I. H. (2008) · Cited by 9,143
On the generalized distance in statistics
Mahalanobis, P. C. (1936) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 6,068
Davis, S., & Mermelstein, P. (1980) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1980.1163420 · Cited by 5,465
A Nonlinear Mapping for Data Structure Analysis
Sammon, J. W. (1969) · IEEE Transactions on Computers · doi:10.1109/t-c.1969.222678 · Cited by 3,380
Estimating Continuous Distributions in Bayesian Classifiers
John, G. H., & Langley, P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4964 · Cited by 2,885
Very Simple Classification Rules Perform Well on Most Commonly Used Datasets
Holte, R. C. (1993) · Machine Learning · doi:10.1023/a:1022631118932 · Cited by 1,791
VII. Mathematical contributions to the theory of evolution.—III. Regression, heredity, and panmixia
Pearson, K. (1896) · Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1896.0007 · Cited by 1,788
Generating Accurate Rule Sets Without Global Optimization
Frank, E., & Witten, I. H. (1998) · Research Commons (University of Waikato) · doi:10.5555/645527.657305 · Cited by 1,269
Research works printed but not linked: 11
- 1992, Correspondence between nasalance scores and listener judgments of hypernasality and hyponasality, Cleft Palate Craniofac J., 29, 346, 10.1597/1545-1569(1992)029<0346:CBNSAL>2.3.CO;2 · doi:10.1597/1545-1569(1992)029<0346:cbnsal>2.3.co;2
- 1967, Predictive coding of speech signals, 360
- 1960, Nasal sounds and nasalization
- A. Zečević, “Ein sprachgestütztes Trainingssystem zur Evaluierung der Nasalität (A speech-supported training system for the evaluation of nasality),” Ph.D. thesis, University of Mannheim, Mannheim, Germany (2002).
- 2008, Speech Recognizer Adaptation
- 2002, Integrated Stochastic Models for Spontaneous Speech Recognition
- 2008, Parallel Robust Speech Recognition
- 2003, Klassifikation von Mustern (Pattern Classification)
- 2009, Automatic Classification of Emotion-Related User States in Spontaneous Children’s Speech
- 2004, Gaikokugohatsuon no jidouhyoutei to yomiayamatta tango no jidoukenshutsu (Automatic evaluation of foreign language pronunciation and automatic recognition of reading errors in vocabulary), 165
- 2005, Data Mining: Practical Machine Learning Tools and Techniques
Cited by
Orozco‐Arroyave, J. R., Belalcázar-Bolaños, E. A., Arias-Londoño, J. D., Vargas‐Bonilla, J. F., Skodda, S., Rusz, J., Daqrouq, K. O., Hönig, F., & Nöth, E. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2467375 · Cited by 147
Self-citationvan der Molen, A. B. M., van Breugel, J. M. M., Janssen, N. G., Admiraal, R. J. C., van Adrichem, L. N. A., Bierenbroodspot, F., Bittermann, D., van den Boogaard, M. H., Broos, P. H., Dijkstra-Putkamer, J. J. M., et al. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10214813 · Cited by 77
Shahin, M., Zafar, U., & Ahmed, B. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2959393 · Cited by 62
Spoer, D. L., Kiene, J., Dekker, P. K., Huffman, S. S., Kim, K. G., Abadeer, A. I., & Fan, K. L. (2022) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000004608 · Cited by 68
Outcome measurements after oral cancer treatment: speech and speech-related aspects—an overview
Schuster, M. E., & Stelzle, F. (2012) · Oral and Maxillofacial Surgery · doi:10.1007/s10006-012-0340-y · Cited by 49
Self-citationVásquez-Correa, J. C., Arias‐Vergara, T., Schuster, M., Orozco‐Arroyave, J. R., & Nöth, E. (2020) · Speech Communication · doi:10.1016/j.specom.2020.07.005 · Cited by 45
Self-citationAutomated speech analysis tools for children’s speech production: A systematic literature review
McKechnie, J., Ahmed, B., Gutierrez-Osuna, R., Monroe, P., McCabe, P., & Ballard, K. J. (2018) · International Journal of Speech-Language Pathology · doi:10.1080/17549507.2018.1477991 · Cited by 54
Cleft prediction before birth using deep neural network
Shafi, N., Bukhari, F., Iqbal, W., Almustafa, K. M., Asif, M., & Nawaz, Z. (2020) · Health Informatics Journal · doi:10.1177/1460458220911789 · Cited by 39
Automatic Evaluation of Hypernasality and Consonant Misarticulation in Cleft Palate Speech
He, L. T., Zhang, J., Liu, Q., Yin, H., & Lech, M. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2333061 · Cited by 37
Artificial Intelligence Applications in Pediatric Craniofacial Surgery
Harrison, L. M., Edison, R. L., & Hallac, R. R. (2025) · Diagnostics · doi:10.3390/diagnostics15070829 · Cited by 34
Automatic Evaluation of Hypernasality Based on a Cleft Palate Speech Database
He, L. T., Zhang, J., Liu, Q., Yin, H., Lech, M., & Huang, Y. (2015) · Journal of Medical Systems · doi:10.1007/s10916-015-0242-2 · Cited by 28
Automatic speech recognition: A primer for speech-language pathology researchers
Keshet, J. (2018) · International Journal of Speech-Language Pathology · doi:10.1080/17549507.2018.1510033 · Cited by 33
Estimation of Hypernasality Scores from Cleft Lip and Palate Speech
Vikram, C. M., Tripathi, A., Kalita, S., & Prasanna, S. R. M. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1631 · Cited by 25
Automatic Hypernasality Detection in Cleft Palate Speech Using CNN
Wang, X., Tang, M., Yang, S., Yin, H., Huang, H., & He, L. T. (2019) · Circuits Systems and Signal Processing · doi:10.1007/s00034-019-01141-x · Cited by 23
Ruff, S., Bocklet, T., Nöth, E., Müller, J., Hoster, E., & Schuster, M. E. (2017) · ORL · doi:10.1159/000479819 · Cited by 24
Self-citationDetection of hypernasality based on vowel space area
Dubey, A. K., Tripathi, A., Prasanna, S. R. M., & Dandapat, S. (2018) · The Journal of the Acoustical Society of America · doi:10.1121/1.5039718 · Cited by 23
Anomaly detection based pronunciation verification approach using speech attribute features
Shahin, M. A., & Ahmed, B. (2019) · Speech Communication · doi:10.1016/j.specom.2019.06.003 · Cited by 24
Ishikawa, K., MacAuslan, J., & Boyce, S. (2017) · The Journal of the Acoustical Society of America · doi:10.1121/1.5009687 · Cited by 21
Acoustic Analysis and Non Linear Dynamics Applied to Voice Pathology Detection: A Review
Orozco‐Arroyave, J. R., Vargas‐Bonilla, J. F., & Delgado‐Trejos, E. (2012) · Recent Patents on Signal Processing · doi:10.2174/2210686311202020096 · Cited by 21
Javid, M. H., Gurugubelli, K., & Vuppala, A. K. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054684 · Cited by 20
Automatic Identification of Articulation Disorders for Arabic Children Speakers
Hanani, A. A., Attari, M., Farakhna, A., Joma'A, A., Hussein, M. S., & Taylor, S. (2016) · Workshop on Child, Computer and Interaction (WOCCI) · doi:10.21437/wocci.2016-6 · Cited by 19
Dubey, A. K., Prasanna, S. R. M., & Dandapat, S. (2019) · The Journal of the Acoustical Society of America · doi:10.1121/1.5134433 · Cited by 21
Zhang, A., Pyon, R. E., Chen, K., & Lin, A. Y. (2023) · FACE · doi:10.1177/27325016231187985 · Cited by 16
Sinusoidal model-based hypernasality detection in cleft palate speech using CVCV sequence
Dubey, A. K., Prasanna, S. R. M., & Dandapat, S. (2020) · Speech Communication · doi:10.1016/j.specom.2020.08.001 · Cited by 17
Automatic detection of glottal stop in cleft palate speech
He, L. T., Zhang, J., Liu, Q., Zhang, J., Yin, H., & Lech, M. (2017) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2017.07.027 · Cited by 14
Machine Learning for Automatic Detection of Velopharyngeal Dysfunction: A Preliminary Report
Lucas, C., Torres‐Guzman, R. A., James, A. J., Corlew, S., Stone, A., Powell, M., Golinko, M. S., & Pontell, M. E. (2024) · Journal of Craniofacial Surgery · doi:10.1097/scs.0000000000010147 · Cited by 13
Automatic Screening Of Children With Speech Sound Disorders Using Paralinguistic Features
Shahin, M. A., Ahmed, B., Smith, D., Duenser, A., & Epps, J. (2019) · Machine learning for signal processing ... · doi:10.1109/mlsp.2019.8918725 · Cited by 22
Zero time windowing analysis of hypernasality in speech of Cleft Lip and palate children
Dubey, A. K., Prasanna, S. R. M., & Dandapat, S. (2016) · National Conference on Communications · doi:10.1109/ncc.2016.7561188 · Cited by 12
Rahman, E., Esfahlani, S. S., Rao, P., Garcia, P., Ioannidis, S., Nosta, J., Rahman, Z., & Webb, W. R. (2025) · European Journal of Plastic Surgery · doi:10.1007/s00238-025-02278-6 · Cited by 11
Kalita, S., Prasanna, S. R. M., & Dandapat, S. (2018) · The Journal of the Acoustical Society of America · doi:10.1121/1.5064463 · Cited by 12
Vowel Onset Point Based Screening of Misarticulated Stops in Cleft Lip and Palate Speech
Mathad, V. C., & Prasanna, S. R. M. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2957887 · Cited by 11
Detection of Nasalized Voiced Stops in Cleft Palate Speech Using Epoch-Synchronous Features
Vikram, C. M., Adiga, N., & Prasanna, S. R. M. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2913089 · Cited by 10
Improving Hypernasality Estimation with Automatic Speech Recognition in Cleft Palate Speech
Song, K., Wan, T., Wang, B., Jiang, H., Qiu, L., Xu, J., Jiang, L., Lou, Q., Yang, Y., Li, D., et al. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-438 · Cited by 13
Automatic evaluation of hypernasality and speech intelligibility for children with cleft palate
He, L. T., Zhang, J., Liu, Q., Yin, H., & Lech, M. (2013) · IEEE Conference on Industrial Electronics and Applications (ICIEA) · doi:10.1109/iciea.2013.6566369 · Cited by 9
Sprachverständlichkeit von Kindern mit bilateralen und unilateralen Lippen-Kiefer-Gaumen-Spalten
Dames, F., Maier, A., Schützenberger, A., Stelzle, F., Holst, A., Nöth, E., Eysholdt, U., & Schuster, M. (2009) · Laryngo-Rhino-Otologie · doi:10.1055/s-0029-1225639 · Cited by 8
Self-citationAutomatic phoneme analysis in children with Cleft Lip and Palate
Bocklet, T., Riedhammer, K., Eysholdt, U., & Nöth, E. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639135 · Cited by 7
Self-citationEvaluation of disfluent speech by means of automatic acoustic measurements
Lustyk, T., Bergl, P., & Čmejla, R. (2014) · The Journal of the Acoustical Society of America · doi:10.1121/1.4863646 · Cited by 7
Enhancement of cleft palate speech using temporal and spectral processing
Sudro, P. N., & Prasanna, S. R. M. (2020) · Speech Communication · doi:10.1016/j.specom.2020.07.002 · Cited by 7
Variational mode decomposition based features for detection of hypernasality in cleft palate speech
Sireesha, K., Dubey, A., Govind, D., Samudravijaya, K., & Gangashetty, S. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106689 · Cited by 8
Hypernasality Detection Using Zero Time Windowing
Dubey, A. K., Prasanna, S. R. M., & Dandapat, S. (2018) · International Conference on Signal Processing and Communications (SPCOM) · doi:10.1109/spcom.2018.8724392 · Cited by 6
From Support Vector Machines to Neural Networks
Alter, N. E., Lucas, C., Torres‐Guzman, R. A., James, A., Stone, A., Powell, M., Corlew, S., Liu, W., Qu, B., Yin, Z., et al. (2025) · Annals of Plastic Surgery · doi:10.1097/sap.0000000000004460 · Cited by 5
Automatic detection of articulations disorders from children’s speech preliminary study
Ramou, N., & Guerti, M. (2014) · Journal of Communications Technology and Electronics · doi:10.1134/s1064226914110187 · Cited by 5
Spectral Analysis Based Objective Assessment of Hypernasality in Cleft Palate Speech: A Review
Dubey, A. K., Prasanna, S. R. M., & Dandapat, S. (2025) · Circuits Systems and Signal Processing · doi:10.1007/s00034-025-03007-x · Cited by 4
The role of AI in the diagnosis of speech and language disorders: A systematic mapping study
Tbaishat, D., Al-Shafei, R., & Odeh, M. (2025) · DIGITAL HEALTH · doi:10.1177/20552076251379769 · Cited by 4
Altalmas, T., Ahmad, S., Hashim, N. N. W. N., Hassan, S. S., & Sediono, W. (2022) · Bulletin of Electrical Engineering and Informatics · doi:10.11591/eei.v11i5.3715 · Cited by 3
Speech assessment following microsurgical soft palate repair
Bschorer, F., Hornig, L., Schön, G., & Bschorer, R. (2023) · Journal of Cranio-Maxillofacial Surgery · doi:10.1016/j.jcms.2023.01.009 · Cited by 3
Automatic detection of consonant omission in cleft palate speech
He, L., Wang, X., Zhang, J., Liu, Q., Yin, H., & Lech, M. (2019) · International Journal of Speech Technology · doi:10.1007/s10772-018-09570-w · Cited by 2
Hypernasality severity analysis using spectral and residual features
Dubey, A. K., Singh, D. K., & Tiwari, B. B. (2021) · International Conference on Internet of Everything, Microwave Engineering, Communication and Networks (IEMECON) · doi:10.1109/iemecon53809.2021.9689095 · Cited by 2
Automatic evaluation of resonance and articulation disorders in cleft palate speech
He, L. T., Tan, J., Hao, H., Tang, M., Yin, H., & Lech, M. (2015) · IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP) · doi:10.1109/chinasip.2015.7230424 · Cited by 3
Acoustic analysis of misarticulated trills in cleft lip and palate children
Vikram, C. M., Macha, S. K., Kalita, S., & Prasanna, S. R. M. (2018) · The Journal of the Acoustical Society of America · doi:10.1121/1.5042339 · Cited by 2
Automatic Severity Evaluation of Articulation Disorder in Speech using Dynamic Time Warping
Devi Krishna, B., Mary, L., & George, A. (2021) · International Conference on Microelectronics, Signals & Systems (ICMSS) · doi:10.1109/icmss53060.2021.9673654 · Cited by 2
Ali, A. S. M., Masaki, K., Hattori, M., Sumita, Y. I., & Wakabayashi, N. (2024) · Journal of Oral Rehabilitation · doi:10.1111/joor.13832 · Cited by 4
Language-independent method for analysis of German stuttering recordings
Lustyk, T., Bergl, P., Haderlein, T., Nöth, E., & Čmejla, R. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-610 · Cited by 2
Self-citationTowards Interpretability of Automatic Phoneme Analysis in Cleft Lip and Palate Speech
Baumann, I., Wagner, D., Schuster, M. E., Nöth, E., & Bocklet, T. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447632 · Cited by 3
Self-citationRoohani, I., Harshbarger, R. J., Padwa, B. L., & Hammoudeh, J. A. (2025) · FACE · doi:10.1177/27325016251384501 · Cited by 1
Multi-Model ASR Integration With Reliability Weighting for Automated Speech Disorder Screening
Sung, S. S., Ha, S., Yoon, T.-J., & So, J. (2026) · IEEE Access · doi:10.1109/access.2026.3667615 · Cited by 1
Speech Disorders Recognition using Speech Analysis
Necibi, K., Bahi, H., & Sari, T. (2012) · IGI Global eBooks · doi:10.4018/978-1-4666-0954-9.ch015 · Cited by 1
Phoneme articulation in students with cerebral palsy and intellectual disability
Đurić-Zdravković, A., Rankovic, S., Japundža‐Milisavljević, M., & Gagić, S. (2017) · Specijalna edukacija i rehabilitacija · doi:10.5937/specedreh16-13247 · Cited by 1
Significance of Source Information in Hypernasality Detection
Dubey, A. K., Singh, D. K., & Tiwari, B. B. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-16-8826-3_7 · Cited by 1
Speech Disorders Recognition using Speech Analysis
Necibi, K., Bahi, H., & Sari, T. (2013) · IGI Global eBooks · doi:10.4018/978-1-4666-4422-9.ch024 · Cited by 1
Künstliche Intelligenz für die Analyse pathologischer Sprache
Bocklet, T., Nöth, E., & Riedhammer, K. (2023) · Sprache · Stimme · Gehör · doi:10.1055/a-2089-5778 · Cited by 1
Self-citationValidierung einer automatischen Analyse der Sprechproben von Kindern mit isolierter Gaumenspalte
Schulz, A.-P., Bocklet, T., Eysholdt, U., Bohr, C., Döllinger, M., & Ziethe, A. (2014) · HNO · doi:10.1007/s00106-013-2825-x · Cited by 3
Self-citationSchuller, B. W., & Batliner, A. M. (2013) · Wiley eBooks · doi:10.1002/9781118706664.ch5 · Cited by 1
Analysis of Feature Extraction Methods for Distinguishing the Speech of Cleft Palate Patients
Kim, S., Kim, W., Kwon, T., Sung, M., & Sung, M. Y. (2015) · Journal of KIISE · doi:10.5626/jok.2015.42.11.1372
Validierung einer automatischen Analyse der Sprechproben von Kindern mit isolierter Gaumenspalte
Mustererkennung, L. F. (2014)
50 Shades of Gray: Effect of the Color Scale for the Assessment of Speech Disorders
Pérez-Toro, P. A., Klumpp, P., Vásquez-Correa, J. C., Schuster, M. E., Nöth, E., Orozco‐Arroyave, J. R., & Arias‐Vergara, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16270-1_29
Self-citationNASAM 2.0: Cleft-Palate Speech Assessment Application
Kurpukdee, N., Thangthai, K., Chunwijitra, V., Chootrakool, P., & Kasuriya, S. (2022) · International Conference on Speech Database and Assessments · doi:10.1109/o-cocosda202257103.2022.9997825
Nila, T. A., Nisha, S. I., & Mim, M. M. (2024) · Social Science Review · doi:10.3329/ssr.v40i2.72206
Roy, M., Reid, R. R., Senkaiahliyan, S., Doria, A. S., Phillips, J. H., Brudno, M., & Singh, D. A. (2024) · Artificial Intelligence Surgery · doi:10.20517/ais.2024.69 · Cited by 1
Chapman, K. L. (2025) · Springer eBooks · doi:10.1007/978-3-031-84663-2_16
Dubey, A. K. (2026) · Circuits Systems and Signal Processing · doi:10.1007/s00034-026-03765-2
Metadata
- Authors
- Maier, A., Hönig, F., Bocklet, T., Nöth, E., Stelzle, F., Nkenke, E., & Schuster, M.
- Affiliations
- Friedrich-Alexander-Universität Erlangen-Nürnberg (DE)
- Universitätsklinikum Erlangen (DE)
- ORCID
- Venue
- The Journal of the Acoustical Society of America
- Publisher
- Acoustical Society of America (ASA)
- Published
- 2009-11-01
- Volume
- 126
- Issue
- 5
- Pages
- 2589-2602
- ISSN
- 0001-4966 (ISSN-L)
- 1520-8524
- OpenAlex
- W2048251069
- Cited by
- 71
- References
- 67
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 6 · 2023: 3 · 2022: 6 · 2021: 3 · 2020: 5 · 2019: 7 · 2018: 8 · 2017: 4 · 2016: 2 · 2015: 5 · 2014: 5 · 2013: 4 · 2012: 3
- FWCI
- 0.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Journal of the Acoustical Society of America (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Deutsche Forschungsgemeinschaft
- Topics
- Health Sciences › Dentistry › Oral Surgery › Cleft Lip and Palate Research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- cleft lip and palate, articulation disorders, speech disorders in children, automatic speech evaluation, speech sound disorders, perceptual speech assessment, interobserver agreement, speech therapy
- MeSH terms
- Algorithms, Articulation Disorders, Child, Cleft Lip, Cleft Palate, Humans, Models, Biological, Phonation, Phonetics, Psycholinguistics, Speech Therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.