Deep learning of brain lesion patterns and user-defined clinical and MRI features for predicting conversion to multiple sclerosis from clinically isolated syndrome
The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
Multiple sclerosis (MS) is a neurological disease with an early course that is characterised by attacks of clinical worsening, separated by variable periods of remission. The ability to predict the risk of attacks in a given time frame can be used to identify patients who are likely to benefit from more proactive treatment. We aim to determine whether deep learning can extract latent MS lesion features that, when combined with user-defined radiological and clinical measurements, can predict conversion to MS (defined with criteria that include new T2 lesions, new T1 gadolinium enhancing lesions and/or new clinical relapse) in patients with early MS symptoms (clinically isolated syndrome), a prodromal stage of MS, more accurately than imaging biomarkers that have been used in clinical studies to evaluate overall disease state, such as lesion volume and brain volume. More specifically, we use convolutional neural networks to extract latent MS lesion patterns that are associated with conversion to definite MS (based on the McDonald 2005 criteria) using lesion masks segmented from baseline MR images. The main challenges are that lesion masks are generally sparse and the number of training samples is small relative to the dimensionality of the images. To cope with sparse voxel data, we propose utilising the Euclidean distance transform (EDT) for increasing information density by populating each voxel with a value indicating distance to the closest lesion boundary. To reduce the risk of overfitting resulting from high image dimensionality, we use a synergistic combination of downsampling, unsupervised pretraining and regularisation during training. A detailed analysis of the impact of EDT and unsupervised pretraining is presented. In contrast to our previous work, which only used automatically learned image features for prediction, we incorporate three user-defined magnetic resonance imaging (MRI) measurements and eight user-defined clinical measurements into the prediction model. In total, the baseline user-defined measurements consist of 11 features. Using the baseline MRI scans and all available measurements from 140 subjects in a sevenfold cross-validation procedure, we demonstrate that our model can achieve an average accuracy rate of 75.0% (SD = 11.3%) in predicting disease activity that is indicative of radiological or clinical conversion to definite MS within two years, which is higher than the 65.0% (SD = 14.6%) that is attained with lesion volume alone. More significantly, our model also outperformed a random forest using all available user-defined measurements (67.9%, SD=10.6%), thereby demonstrating the potential benefit of automatic extraction of latent lesion features by deep learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Radiological data has been combined with other data types for predictive AI systems in various clinical disciplines like oncology21–24 or neurology25,26.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [26]
In the citing paper
“Yoo et al. further compared their joint fusion model to a late fusion model and achieved a 0.02 increase in Area Under Receiver Operating Characteristic Curve (AUROC).”
From Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines (2020) · cites this paper as [38]
In the citing paper
“To cater to the differences between the dimensionality and dynamic range between the imaging and clinical features, Yoo et al. replicated and scaled their clinical features before fusion and they observed improvements in performances.”
From Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines (2020) · cites this paper as [38]
In the citing paper
“Kawahara et al., Yala et al. and Yoo et al. reported an improvement in performance using fusion compared to image-only models (Table 1).”
From Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines (2020) · cites this paper as [38]
In the citing paper
“Spasov et al.36, Yala et al.37, and Yoo et al.38 implemented CNNs to learn image features and fused these feature representations with clinical features before feeding them into a feed-forward neural network.”
From Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines (2020) · cites this paper as [38]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Rating neurologic impairment in multiple sclerosis: An expanded disability status scale (EDSS)
Kurtzke, J. F. (1983) · Neurology · doi:10.1212/wnl.33.11.1444 · Cited by 15,011
Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria
Polman, C. H., Reingold, S. C., Banwell, B., Clanet, M., Cohen, J. A., Filippi, M., Fujihara, K., Havrdova, E., Hutchinson, M., Kappos, L., et al. (2011) · Annals of Neurology · doi:10.1002/ana.22366 · Cited by 9,802
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
Diagnostic criteria for multiple sclerosis: 2005 revisions to the “McDonald Criteria”
Polman, C. H., Reingold, S. C., Edan, G., Filippi, M., Hartung, H., Kappos, L., Lublin, F. D., Metz, L. M., McFarland, H. F., O'Connor, P. W., et al. (2005) · Annals of Neurology · doi:10.1002/ana.20703 · Cited by 5,017
Self-citationConvolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277 · cited 1×
“It has been shown that unsupervised pretraining can improve the optimisation performance of supervised deep networks, especially when training sets are limited, which often happens in the medical imaging domain (Tajbakhsh et al. 2016), but the gains are dependent on data properties.”
Background
Research works printed but not linked: 6
- Bishop CM, 2006, Mach Learn, 128, 1
- Maas AL, 2013, International Conference on Machine Learning, 30
- Orr GB, 2012, Neural networks: tricks of the trade
- Srivastava N, 2014, J Mach Learn Res, 15, 1929
- Theano Development Team, 2016, Theano: a Python framework for fast computation of mathematical expressions
- Van Der Maaten L, 2014, J Mach Learn Res, 15, 3221
Cited by
Huang, S.-C., Pareek, A., Seyyedi, S., Banerjee, I., & Lungren, M. P. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00341-z · Cited by 934
“Yoo et al. 38 took the mean of the predicted probabilities from two single modality models as the fi nal prediction.”
BackgroundMethodInfluentialA review on deep learning approaches in healthcare systems: Taxonomies, challenges, and open issues
Shamshirband, S., Fathi, M., Dehzangi, A., Chronopoulos, A. T., & Alinejad-Rokny, H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103627 · Cited by 323
Chen, J., Yi, C., Okegbile, S. D., Cai, J., & Shen, X. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3308717 · Cited by 254
Review of multimodal machine learning approaches in healthcare
Krones, F., Marikkar, U., Parsons, G., Szmul, A., & Mahdi, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102690 · Cited by 235
Pahud de Mortanges, A., Luo, H., Shu, S. Z., Kamath, A., Suter, Y., Shelan, M., Pöllinger, A., & Reyes, M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01190-w · Cited by 93
Metadata
- Authors
- Yoo, Y., Tang, L. Y. W., Li, D. K. B., Metz, L., Kolind, S., Traboulsee, A. L., & Tam, R. C.
- Affiliations
- University of British Columbia (CA)
- University of Calgary (CA)
- Corresponding author
- Yoo, Youngjin
- ORCID
- Venue
- Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization
- Publisher
- Informa UK Limited
- Published
- 2017-08-09
- Volume
- 7
- Issue
- 3
- Pages
- 250-259
- ISSN
- 2168-1163 (ISSN-L)
- 2168-1171
- OpenAlex
- W2742947264
- Cited by
- 67
- References
- 37
- Citations by year
- 2026: 5 · 2025: 13 · 2024: 15 · 2023: 13 · 2022: 9 · 2021: 7 · 2020: 4 · 2019: 1
- FWCI
- 1.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization (publishedVersion)
- Funders
- Multiple Sclerosis Society of Canada (Identifier: NCT00666887, NCT00666887)
- Natural Sciences and Engineering Research Council of Canada (RGPIN 402202-12)
- endMS Doctoral Studentship Award from the Multiple Sclerosis Society of Canada (NCT00666887, MS Society Project Number 2739)
- Topics
- Health Sciences › Medicine › Clinical Neurology › Multiple Sclerosis Research Studies
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Keywords
- multiple sclerosis, clinically isolated syndrome, conversion to multiple sclerosis, convolutional neural networks, Euclidean distance transform, unsupervised pre-training, MRI lesion volume, disease activity prediction, lesion masks, overfitting, McDonald criteria, recurrence prediction, random forest comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep learning of brain lesion patterns and user-defined clinical and MRI features for predicting conversion to multiple sclerosis from clinically isolated syndrome. https://app.cochat.ai/papers/10.1080/21681163.2017.1356750
- The paper
- Yoo, Y., Tang, L. Y. W., Li, D. K. B., Metz, L., Kolind, S., Traboulsee, A. L., & Tam, R. C. (2019). Deep learning of brain lesion patterns and user-defined clinical and MRI features for predicting conversion to multiple sclerosis from clinically isolated syndrome. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization. https://doi.org/10.1080/21681163.2017.1356750
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.