Static Tamil Sign Language Recognition using Mediapipe Hand Landmark Detection and Machine Learning

Abstract

Sign language is essential for communication among the deaf and hard-ofhearing communities, addressing the needs of over 430 million people globally, including 34 million children. By 2050, this number is expected to surpass 700 million, emphasizing the critical role of sign language. Extensive research has been undertaken in the field of sign language recognition (SLR) for several languages, including American, Australian, Korean, and Japanese. However, Tamil sign language, which possesses a rich linguistic heritage encompassing 247 characters, has remained largely underexplored in focused academic studies. This research addresses this gap by presenting an innovative method for static Tamil Sign Language Recognition through the application of the MediaPipe Hand Landmark Detection framework combined with machine learning techniques. Our methodology involves using an existing dataset of approximately 254,147 samples representing all 247 Tamil letters. The MediaPipe framework was employed to extract 21 key landmarks from each hand in the dataset. The extracted landmarks were used as features to train various classifiers, including Random Forest, XGBoost, Gradient Boosting, and Support Vector Machine (SVM). These algorithms were applied to the model, and their performance was compared. Comparative analysis of these classifiers identified the Random Forest algorithm as the most effective, achieving a classification accuracy of 96.45%. This high accuracy demonstrates the potential of MediaPipe and Random Forest in recognizing static Tamil “sign language” gestures efficiently and in real time. To our knowledge, this research presents the first comprehensive system for static Tamil Sign Language recognition, offering a significant advancement in SLR by leveraging modern technologies. The findings highlight the feasibility of using MediaPipe Hand Landmark Detection and machine learning to develop robust, real-time sign language recognition systems, providing a vital tool for improving communication accessibility for the Tamil-speaking deaf community