Scaling behaviors of deep learning and linear algorithms for the prediction of stroke severity

Abstract

Introduction: Deep learning has allowed for remarkable progress in many medical scenarios. Since deep learning prediction models often require 10 5 -10 7 examples, it is currently unknown whether deep learning can also enhance predictions of symptoms post-stroke in real-world samples of stroke patients that are often several magnitudes smaller. Such stroke outcome predictions however could be particularly instrumental in guiding acute clinical and rehabilitation care decisions. We here compared the capacities of classically used linear and novel deep learning algorithms in their prediction of stroke severity. Methods: Our analyses relied on a total of 1,430 patients assembled from the MRI-GENIE collaboration and a Massachusetts General Hospital-based study. The outcome of interest was NIHSS-based stroke severity in the acute phase after ischemic stroke onset, which we predict by means of MRI-derived lesion location. We automatically derived lesion segmentations from diffusion-weighted clinical MRI scans, performed spatial normalization and included a principal component analysis (PCA) step, retaining 95% of the variance of the original data. We then repeatedly separated a train, validation, and test set to investigate the effects of sample size, we subsampled the train set to 100, 300, and 900 and trained the algorithms to predict the NIHSS score for each sample size with regularized linear regression and an 8-layered neural network. We selected hyperparameters on the validation set. We evaluated model performance based on the explained variance (R-squared) in the test set. Results: While linear regression performed significantly better for a sample size of 100 patients, deep learning started to significantly outperform linear regression when trained on 900 patients. Average prediction performance improved by ∼20% when increasing the sample size 9x (maximum for 100 patients: 0.279 ± 0.005 (R 2 , 95% confidence interval), 900 patients: 0.337 ± 0.006). Conclusions: For sample sizes of 900 patients, deep learning showed a higher prediction performance than typically employed linear methods. These findings suggest the existence of non-linear relationships between lesion location and stroke severity that can be utilized for an improved prediction performance for larger sample sizes.