LBPV for Recognition of Sign Language at Sentence Level: An Approach Based on Symbolic Representation

Recognition of signs made by deaf people to produce equivalent textual description for normal people to communicate with deaf people is an essential and challenging task for the pattern recognition and image processing research community. Many researchers have made an attempt to standardize and to propose a sign language recognition system. To the best our knowledge, according to the literature survey, most of the work reported has concentrated at the fingerspelling level or at the word level, and less work at the sentence level has been reported. As sign languages are very abstract, fingerspelling or word level interpretation of signs seems to be a tedious and cumbersome task. Although existing research in sign language recognition is active and extensive, it still remains a challenge to achieve accurate recognition and interpretation of signs at the sentence level. In this paper, we made an attempt to address this problem by proposing an approach that exploits the texture description technique and symbolic data analysis concept to characterize and effectively represent a sign, taking into account the intra-class variations due to different signers or the same signers at different instances of time. In order to study the efficacy of the proposed approach, extensive experiments were carried out on a considerably large database of Indian sign language created by us. The experimental results demonstrated that the proposed method has shown good recognition performance in terms of F-measure rates.