MudraGyaan: A Novel Feature Extraction Algorithm for Machine Learning-Based Bharatanatyam Mudra Classification

• Published in: 2024 International Conference on Advancement in Renewable Energy and Intelligent Systems (AREIS) pp. 1 - 6
• Main Authors: Baskar, Sarvesh, Jino Hans, W., Anuprapaa, V R, Sherlin Solomif, V., Arthi, R
• Format: Conference Proceeding
• Language: English
• Published: IEEE 05.12.2024
• Subjects: Bharatanatyam; Classification algorithms; Feature extraction; Hands; Human Computer Interaction; Humanities; Landmarking; Machine Learning; Machine learning algorithms; Measurement; Mediapipe; Mudra; Random Forest; Random forests; Real-time systems; Renewable energy sources; Streaming media
• DOI: 10.1109/AREIS62559.2024.10893664

Bharatanatyam is an extremely narrative traditional dance of Tamil Nadu, India. The narratives are expressed by the dancer mainly through hand gestures called Mudras. Each of these mudras corresponds to their own meanings. In the current work, a real-time Mudra classification, a kind of hand gesture classification is carried out by developing a custom-built dataset of Bharatanatyam Gestures. This paper proposes a MudraGyaan algorithm, a landmarking-based approach to classify the mudras by incorporating logic-based feature extraction. These features are extracted based on finger position and orientation. The finger position states whether the finger is closed or half-open and fully open and orientation explains how the fingers are oriented. Using Google Mediapipe Library landmarking is performed and the points of interest are calculated using distance metrics. Random forest Classifier is used to classify the mudras based on points of interest. Results reveal that a 99.5% accuracy is obtained for Mudra Classification.