Evaluating machine learning algorithms estimating tremor severity ratings on the Bain–Findley scale

Abstract

Tremor is a debilitating symptom of some movement disorders. Effective treatment, such as deep brain stimulation (DBS), is contingent upon frequent clinical assessments using instruments such as the Bain–Findley tremor rating scale (BTRS). Many patients, however, do not have access to frequent clinical assessments. Wearable devices have been developed to provide patients with access to frequent objective assessments outside the clinic via telemedicine. Nevertheless, the information they report is not in the form of BTRS ratings. One way to transform this information into BTRS ratings is through linear regression models (LRMs). Another, potentially more accurate method is through machine learning classifiers (MLCs). This study aims to compare MLCs and LRMs, and identify the most accurate model that can transform objective tremor information into tremor severity ratings on the BTRS. Nine participants with upper limb tremor had their DBS stimulation amplitude varied while they performed clinical upper-extremity exercises. Tremor features were acquired using the tremor biomechanics analysis laboratory (TREMBAL). Movement disorder specialists rated tremor severity on the BTRS from video recordings. Seven MLCs and 6 LRMs transformed TREMBAL features into tremor severity ratings on the BTRS using the specialists’ ratings as training data. The weighted Cohen’s kappa ( κ w ) defined the models’ rating accuracy. This study shows that the Random Forest MLC was the most accurate model ( κ w = 0.81) at transforming tremor information into BTRS ratings, thereby improving the clinical interpretation of tremor information obtained from wearable devices.