A Deep Learning based approach to classification of Baby Sign Language images

Baby Sign Language is used by hearing parents to hearing infants as a preverbal communication which reduce frustration of parents and accelerated learning in babies, increases parent-child bonding, and lets babies communicate vital information, such as if they are hurt or hungry is known as a Baby Sign Language . In the current research work, a study of various existing sign language has been carried out as literature and then after realizing that there is no dataset available for Baby Sign Language, we have created a static dataset for 311 baby signs, which were classified using a MobileNet V1, pretrained Convolution Neural Network [CNN].The focus of the paper is to analyze the effect of Gradient Descent based optimizers, Adam and its variants, Rmsprop optimizers on fine-tuned pretrained CNN model MobileNet V1 that has been trained using customized dataset. The optimizers are used to train and test on MobileNet for 100 epochs on the dataset created for 311 baby Signs. These 10 optimizers Adadelta, Adam, Adamax, SGD, Adagrad, RMSProp were compared based on their processing time.

Employment of baby sign language in interactive learning for Hearing-impaired people in Malaysia

<span lang="EN-GB">Communication is very important for people including adults and babies to exchange their thoughts, views and information. They can express their thoughts verbally and non-verbally. Baby sign language is non-verbal type that consists of a set of standard and organised form of hand gestures motion to represent different word. This language allows the children who have hearing impairment problem to communicate with others. In addition, it can also be taught to normal babies to convey their need before they have the ability to speak. It is very challenging for adults to understand babies and children with hearing impaired problem. Thus, a real-time hand gesture recognition system has been developed for adult to interactively learn the baby sign language. Firstly, the face region was detected in order to use the skin colour model of the face to detect the hand. After that, the hand shape was determined by finding the palm centre and convex hull. Convexity defects which representing gaps between the fingers were used to determine the number of fingertips whereas the movement of the hand was detected based on motion analysis. By setting the background colour to blue and adding extra light in front of the laptop, the system successfully obtained an accuracy higher than 85% rate for sign “mom”, “eat” and “milk” when tested on Malay, Chinese and Indian races in Malaysia.</span>