Assistive Technology based Solution for Hearing Impairment using Smartphone

Developing a system for sign language recognition becomes essential for the deaf as well as a mute person. The recognition system acts as a translator between a disabled and an able person. This eliminates the hindrances in the exchange of ideas. Most of the existing systems are very poorly designed with limited support for the needs of their day to day facilities. The proposed system embedded with gesture recognition capability has been introduced here which extracts signs from a video sequence and displays them on screen. On the other hand, a speech to text as well as text to speech system is also introduced to further facilitate the grieved people. To get the best out of a human-computer relationship, the proposed solution consists of various cutting-edge technologies and Machine Learning based sign recognition models that have been trained by using TensorFlow and Keras library. The proposed architecture works better than several gesture recognition techniques like background elimination and conversion to HSV

Design and implementation of children’s gesture education games based on AI gesture recognition technology

In order to cultivate children’s imagination and creativity in the cognitive process, combined with the traditional hand shadow game, a children’s gesture education game based on AI gesture recognition technology is designed and developed. The game uses unity development platform, with children’s digital gesture recognition as the content, designs and implements the basic functions involved in the game, including AI gesture recognition function, character animation function, interface interaction function, AR photo taking function and question answering system function. The game is finally released on the mobile terminal. Players can recognize gestures through mobile cameras, interact with virtual cartoon characters in the game, watch cartoon character animation, understand popular science knowledge, and complete the answers in the game. The educational games can better assist children to learn digital gestures, enrich children’s ways of cognition, expand children’s imagination, and let children learn easily with happy educational games.