Quantifying Device Type and Handedness Biases in a Remote Parkinson's Disease AI‐Powered Assessment

Abstract

Abstract Early detection of Parkinson’s Disease (PD) can enable early access to care, improving patient outcomes. We investigate the use of machine learning to predict PD using data recorded from a web application measuring structured mouse and keypress data through tests assessing finger and hand movement patterns. We evaluate the impact of demographic bias and bias related to device type and handedness, which are particularly relevant to our application. We collected data from 251 participants (99 PD, 152 Non-PD). Using a random forest model, we observed an 84% F1 score, 86% sensitivity, and 92% specificity. When examining only F1-score differences across groups, no significant bias appears. However, conducting a more in-depth analysis using algorithmic fairness metrics uncovers bias regarding the positive prediction and error rates. In particular, we found that sex and ethnicity have no statistically significant impact on receiving a PD prediction. However, we observe biases regarding device type and dominant hand in terms of receiving a PD diagnosis, as evidenced by disparate impact and equalized odds fairness metrics. This work demonstrates that remote digital health diagnostics using consumer devices like desktops or laptops can exhibit nontraditional yet significant biases concerning understudied factors in algorithmic fairness such as device type and handedness.