SignPro-A Smart Tool for Deaf Bodied

Abstract

Discourse-handicapped (speech-disabled) people communicate in sign language. These people have trouble socializing with the able-bodied due to communication barriers. There is need of a system through which the sign language is interpretable for able-bodied, because most of the able-bodied do not know their gesture language. Two different methodologies are used to recognize hand gestures. In first approach, 3D coordinates-based multi-level multi-cascation approach will be used on 3d based Hand Gesture Recognition part. The signs and as well as emotions are captured by using a camera. The 3d hand pose is estimated from regular RGB frames. This approach solves the problem of dynamic signs that depends on 3d vectored dimensions (x, y, z) environment as well as the signs that are not static but require the specific movements as well to differentiate different signs that may have certain signs conflict with each other but differentiate on basis of sequence of movement. The 2D approach specifically solves the problem of background skin color conflicted environment by taking a Deep Neural Network approach as well as recognizes the signs that are not static but require the specific movements as well to differentiate different signs that may have certain signs conflicts with each other but differentiate on basis of sequence of movement. This system provides opportunities to people with speech disabilities able communicate and engage with able-bodied in real-time communication.