A triplet multimodel transfer learning network for speech disorder screening of Parkinson’s disease
Open accessAbstract
Deterioration in the quality of a person’s voice and speech is an early sign of Parkinson’s disease (PD). Although a number of computer-based methods have been invested to use patients’ speech for early diagnosis of Parkinson’s disease, they only focus on a fixed pronunciation test, such as the subjects’ monosyllabic pronunciation is analyzed to determine whether they have potential possibility of PD. Moreover, only using traditional speech analysis methods to extract single-view speech features cannot provide a comprehensive feature representation. This paper is dedicated to the study of various pronunciation tests for patients with PD, including the pronunciation of five monosyllabic vowels and a spontaneous dialogue. A triplet multimodel transfer learning network is designed and proposed for identifying subjects with PD in these two groups of tests. First, multisource data extract mel frequency cepstrum coefficient (MFCC) features of speech for preprocessing. Subsequently, a pretrained triplet model represents features from three dimensions as the upstream task of the transfer learning framework. Finally, the pretrained model is reconstructed as a novel model that integrates the triplet model, temporal model, and auxiliary layer as the downstream task, and weights are updated through fine-tuning to identify abnormal speech. Experimental results show that the highest PD detection rates in the two groups of tests are 99% and 90% , respectively, which outperform a large number of internationally popular pattern recognition algorithms and serve as a baseline for other academic researchers in this field.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing acoustic-based approaches such as NAA and TMTLN fail to simultaneously characterize articulation, cognition, and behavior issues in speech disorders because they focus on low-level speech abnormalities and neglect neurolinguistic semantic inconsistency.
“The limitation of existing acoustic-based approaches such as NAA and TMTLN for simultaneously characterizing the issues related to articulation, cognition, and behavior associated with speech disorders is because of their focus on low-level speech abnormalities while failing to consider neurolinguistic inconsistency related to semantics.”
From NSX-Net: A Neurolinguistic and Acoustic Multimodal Deep Learning Framework for Speech Disorder Classification (2026) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Rusz, J., Čmejla, R., Ruzickova, H., & Růžička, E. (2011) · The Journal of the Acoustical Society of America · doi:10.1121/1.3514381 · Cited by 498 · see citing works
Data-Driven Structural Health Monitoring Using Feature Fusion and Hybrid Deep Learning
Dang, H. V., Tran-Ngoc, H., Nguyen, T. V., Bui-Tien, T., De Roeck, G., & Nguyen, H. X. (2021) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.3034401 · Cited by 170 · see citing works
LIGHT-SERNET: A Lightweight Fully Convolutional Neural Network for Speech Emotion Recognition
Aftab, A., Morsali, A., Ghaemmaghami, S., & Champagne, B. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746679 · Cited by 130 · see citing works
Ye, J., Wen, X.-C., Wei, Y., Xu, Y. N., Liu, K., & Shan, H. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096370 · Cited by 122 · see citing works
Speech dysfunction in early Parkinson's disease
Stewart, C. F., Winfield, L. M., Hunt, A. L., Bressman, S., Fahn, S., Blitzer, A., & Brin, M. F. (1995) · Movement Disorders · doi:10.1002/mds.870100506 · Cited by 113 · see citing works
Shahamiri, S. R., & Salim, S. S. (2014) · Advanced Engineering Informatics · doi:10.1016/j.aei.2014.01.001 · Cited by 84 · see citing works
Multimodal Gait Recognition for Neurodegenerative Diseases
Zhao, A., Li, J., Dong, J., et al. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3056104 · Cited by 83 · see citing works
Self-citationRusz, J., Čmejla, R., Růžičková, H., et al. (2012) · Journal of Neural Transmission · doi:10.1007/s00702-012-0853-4 · Cited by 82 · see citing works
Ferreira, M. I. A. S. N., Barbieri, F. A., Moreno, V. C., et al. (2022) · Gait & Posture · doi:10.1016/j.gaitpost.2022.08.014 · Cited by 72 · see citing works
Masked Face Recognition using ResNet-50Preprint
Mandal, B., Okeukwu, A., & Theis, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08997 · Cited by 70 · see citing works
Attention-based BiLSTM models for personality recognition from user-generated content
Zhou, L., Zhang, Z., Zhao, L., & Yang, P. (2022) · Information Sciences · doi:10.1016/j.ins.2022.03.038 · Cited by 68 · see citing works
Ye, J., Wen, X.-C., Wang, X.-Z., Xu, Y. D., Luo, Y., Wu, C., Chen, L., & Liu, K. (2022) · Speech Communication · doi:10.1016/j.specom.2022.07.005 · Cited by 64 · see citing works
Privacy-Preserving Outsourced Speech Recognition for Smart IoT Devices
Ma, Z., Liu, Y., Liu, X., Ma, J., & Li, F. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2917933 · Cited by 59 · see citing works
Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition
Hernandez, A., Pérez-Toro, P. A., Nöth, E., Orozco‐Arroyave, J. R., Maier, A., & Yang, S. H. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10674 · Cited by 57 · see citing works
Severity level diagnosis of Parkinson’s disease by ensemble K-nearest neighbor under imbalanced data
Zhao, H., Wang, R., Lei, Y., Liao, W., Cao, H., & Cao, J. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116113 · Cited by 56 · see citing works
Associated Spatio-Temporal Capsule Network for Gait Recognition
Zhao, A., Dong, J., Li, J., Qi, L., & Zhou, H. (2021) · arXiv · doi:10.48550/arxiv.2101.02458 · Cited by 55 · see citing works
Self-citationAssociated Spatio-Temporal Capsule Network for Gait Recognition
Zhao, A., Dong, J., Li, J., et al. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3060280 · Cited by 55 · see citing works
Self-citationDeep dual-side learning ensemble model for Parkinson speech recognition
Ma, J., Zhang, T., Li, Y., Zhou, L., Qin, L., Zeng, Y., Wang, P., & Lei, Y. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.102849 · Cited by 46 · see citing works
Jaeger, H., Trivedi, D., & Stadtschnitzer, M. (2019) · Figshare · doi:10.5281/zenodo.2867215 · Cited by 45 · see citing works
Speech difficulties in early de novo patients with Parkinson's disease
Polychronis, S., Niccolini, F., Pagano, G., Yousaf, T., & Politis, M. (2019) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2019.04.026 · Cited by 42 · see citing works
Fault Identification Based on PD Ultrasonic Signal Using RNN, DNN and CNN
Zhang, Q., Lin, J., Song, H., et al. (2018) · 2018 Condition Monitoring and Diagnosis (CMD) · doi:10.1109/cmd.2018.8535878 · Cited by 38 · see citing works
Liu, Y., Li, Y., Tan, X., Wang, P., & Zhang, Y. (2020) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2020.102165 · Cited by 38 · see citing works
Liu, R., Wang, Z., Qiu, S., Zhao, H., Wang, C., Shi, X., & Lin, F. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3174249 · Cited by 37 · see citing works
Xu, M., Du, J., Xue, Z., et al. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22846 · Cited by 36 · see citing works
ResNet-Based Parkinson's Disease Classification
Ariss, O. E., & Hu, K. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3193651 · Cited by 31 · see citing works
Identifying Parkinson’s Disease using Multimodal Approach and Deep Learning
Mohaghegh, M., & Gascon, J. (2021) · International Conference on Innovative Technologies in Intelligent Systems and Industrial Applications (CITISIA) · doi:10.1109/citisia53721.2021.9719945 · Cited by 30 · see citing works
Transferable Self-Supervised Instance Learning for Sleep Recognition
Zhao, A., Wang, Y., & Li, J. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3176751 · Cited by 30 · see citing works
Self-citationZhao, A., Wu, H., Chen, M., & Wang, N. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/9921809 · Cited by 30 · see citing works
Self-citationWen, X.-C., Ye, J., Luo, Y., Xu, Y. N., Wang, X.-Z., Wu, C.-L., & Liu, K. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/320 · Cited by 29 · see citing works
Classification of multiple PD Sources by Signal Features and LSTM Networks
Adam, B., & Tenbohlen, S. (2018) · IEEE International Conference on High Voltage Engineering and Application (ICHVE) · doi:10.1109/ichve.2018.8642226 · Cited by 25 · see citing works
Discriminating Parkinson diseased and healthy people using modified MFCC filter bank approach
Upadhya, S. S., Cheeran, A. N., & Nirmal, J. H. (2019) · International Journal of Speech Technology · doi:10.1007/s10772-019-09647-0 · Cited by 24 · see citing works
Italian Parkinson's Voice and Speech
Dimauro, G., & Girardi, F. (2019) · CINECA IRIS Institutional Research Information System (University of Bari Aldo Moro) · doi:10.21227/aw6b-tg17 · Cited by 23 · see citing works
Ullah, F., Naeem, M. R., Naeem, H., Cheng, X., & Alazab, M. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22813 · Cited by 23 · see citing works
Towards an accurate radar waveform recognition algorithm based on dense CNN
Si, W., Wan, C., & Zhang, C. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09490-5 · Cited by 22 · see citing works
Self-attention-based BiGRU and capsule network for named entity recognitionPreprint
Deng, J., Cheng, L., & Wang, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00735 · Cited by 18 · see citing works
Enhancing Speech Recognition for Parkinson’s Disease Patient Using Transfer Learning Technique
Yu, Q., Ma, Y., & Li, Y. (2021) · Journal of Shanghai Jiaotong University (Science) · doi:10.1007/s12204-021-2376-3 · Cited by 14 · see citing works
Medina, C. A., Vargas, E., Munger, S. J., & Miller, J. E. (2022) · PLoS ONE · doi:10.1371/journal.pone.0265604 · Cited by 13 · see citing works
Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNet
Dhakal, M., Chhetri, A., Gupta, A., et al. (2022) · International Conference on Inventive Computation Technologies (ICICT) · doi:10.1109/icict54344.2022.9850832 · Cited by 12 · see citing works
Wen, X.-C., Ye, J., Luo, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.10644 · Cited by 10 · see citing works
Early Diagnosis of Parkinson's Disease with Speech Pronunciation Features Based on XGBoost Model
Wang, X., Chen, X., Wang, Q., et al. (2022) · IEEE International Conference on Software Engineering and Artificial Intelligence (SEAI) · doi:10.1109/seai55746.2022.9832191 · Cited by 10 · see citing works
Vásquez-Correa, J. C., García-Ospina, N., Orozco‐Arroyave, J. R., Cerňak, M., & Nöth, E. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00794-2_49 · Cited by 7 · see citing works
ResCNN: An alternative implementation of Convolutional Neural Networks
Bose, S., & Dey, A. (2021) · IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON) · doi:10.1109/upcon52273.2021.9667654 · Cited by 4 · see citing works
Fine-grained Early Frequency Attention for Deep Speaker Recognition
Hajavi, A., & Etemad, A. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892054 · Cited by 4 · see citing works
A multi-level feature attention network for COVID-19 detection based on multi-source medical images
Zhao, A., Wu, H., Chen, M., & Wang, N. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-18014-w · Cited by 4 · see citing works
Self-citationSpeech Recognition for Power Customer Service Based on DNN and CNN Models
Yin, Y., Wu, M., Wang, X., et al. (2022) · Communications in computer and information science · doi:10.1007/978-981-19-2266-4_35 · Cited by 3 · see citing works
Qin, J., Liu, T., Wang, Z., et al. (2022) · Communications in computer and information science · doi:10.1007/978-981-19-5194-7_21 · Cited by 3 · see citing works
Deep Double-Side Learning Ensemble Model for Few-Shot Parkinson Speech RecognitionPreprint
Li, Y., Zhou, L., Qin, L., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11593 · Cited by 2 · see citing works
Extensive Analysis of Global Presidents’ Speeches Using Natural Language
Nivash, S., Ganesh, E. N., Harisudha, K., et al. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-981-16-5157-1_65 · Cited by 2 · see citing works
Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech RecognitionPreprint
Hernandez, A., Pérez-Toro, P. A., Nöth, E., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.01670 · Cited by 2 · see citing works
Cesarelli, G., Donisi, L., Coccia, A., Amitrano, F., Biancardi, A., Lanzillo, B., & D’Addio, G. (2022) · IEEE International Symposium on Medical Measurements and Applications (MeMeA) · doi:10.1109/memea54994.2022.9856460 · Cited by 2 · see citing works
Fine-grained Early Frequency Attention for Deep Speaker RecognitionPreprint
Hajavi, A., & Etemad, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.10006 · Cited by 1 · see citing works
Pati, B. M., Mahanta, M., & Taparugssanagorn, A. (2022) · International Journal of Sensor Networks · doi:10.1504/ijsnet.2022.10049990 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- S. Woo, J. Park, J. Y. Lee, and I. S. Kweon, "Cbam: con- volutional block attention module," in European Conference on Computer Vision, Springer, Singapore, 2018.
Cited by
Sorted by citations
Nadeem, A., Khan, M. Z., Sirshar, M., & Thirunavukkarasu, U. (2026) · Diagnostics · doi:10.3390/diagnostics16152377
- Authors
- Zhao, A., Wang, N., Niu, X., Chen, M., & Wu, H.
- Affiliations
- Qingdao University (CN)
- Corresponding author
- Zhao, Aite
- Venue
- International Journal of Intelligent Systems
- Publisher
- Wiley
- Published
- 2024-03-20
- Volume
- 2024
- Pages
- 1-20
- ISSN
- 0884-8173 (ISSN-L)
- 1098-111X
- OpenAlex
- W4392977357
- Cited by
- 16
- References
- 53
- Citations by year
- 2026: 4 · 2025: 11 · 2024: 1
- FWCI
- 4.56
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- International Journal of Intelligent Systems (publishedVersion, cc-by)
- Funders
- National Natural Science Foundation of China (62106117, 2022M711741, ZR2021QF084, 2023KJ365)
- China Postdoctoral Science Foundation (62106117, 2022M711741, ZR2021QF084, 2023KJ365)
- Natural Science Foundation of Shandong Province (62106117, 2022M711741, ZR2021QF084, 2023KJ365)
- Shandong Province Higher Education Institutions Youth Innovation and Technology Support Program (62106117, 2022M711741, ZR2021QF084, 2023KJ365)
- Topics
- Health Sciences › Medicine › Physiology › Voice and Speech Disorders
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- Parkinson's disease, early diagnosis, triplet loss, spontaneous dialogue, Mel-frequency cepstral coefficients, speech feature extraction, fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review