Arabic sign language recognition using the leap motion controller

Abstract

Sign language is important for facilitating communication between hearing impaired and the rest of society. Two approaches have traditionally been used in the literature: image-based and sensor-based systems. Sensor-based systems require the user to wear electronic gloves while performing the signs. The glove includes a number of sensors detecting different hand and finger articulations. Image-based systems use camera(s) to acquire a sequence of images of the hand. Each of the two approaches has its own disadvantages. The sensor-based method is not natural as the user must wear a cumbersome instrument while the imagebased system requires specific background and environmental conditions to achieve high accuracy. In this paper, we propose a new approach for Arabic Sign Language Recognition (ArSLR) which involves the use of the recently introduced Leap Motion Controller (LMC). This device detects and tracks the hand and fingers to provide position and motion information. We propose to use the LMC as a backbone of the ArSLR system. In addition to data acquisition, the system includes a preprocessing stage, a feature extraction stage, and a classification stage. We compare the performance of Multilayer Perceptron (MLP) neural networks with the Nave Bayes classifier. Using the proposed system on the Arabic sign alphabets gives 98% classification accuracy with the Nave Bayes classifier and more than 99% using the MLP.