Filipino Sign Language Hand Gesture Recognition Using MediaPipe and Machine Learning

Lorraine Kaye M. Pilare; Junejay Christian A. Mahinay; Augustine Clein C. Degamo; Brill Nash C. Piner

Lorraine Kaye M. Pilare Engineering Department, University of San Jose-Recoletos, Philippines
Junejay Christian A. Mahinay Engineering Department, University of San Jose-Recoletos, Philippines
Augustine Clein C. Degamo Engineering Department, University of San Jose-Recoletos, Philippines
Brill Nash C. Piner Engineering Department, University of San Jose-Recoletos, Philippines

Abstract

Purpose –This study develops a Filipino sign language recognition system that can recognize sign language composed of three (3) basic words and twenty-seven (27) phrases.

Method – The proposed sign language recognition method converts sign language into text through several steps. First, video input is captured using a webcam. MediaPipe then extracts features of both the left and right hands from the video. An LSTM algorithm is trained to recognize patterns in these hand features, translating sign language into text accurately. The translated text is displayed on a monitor. A total of 900 data samples were used, with an output shape of (900, 30, 128) for a video containing 30 frames and 64 hand landmarks per frame.

Results – The proponents performed test cases to confirm that the system met the necessary quality standards, using three different dataset splits. The highest training accuracy percentage was 98.41% with 70% of training and 30% samples. The highest testing accuracy percentage was 98.89% with an even 50% split between training and testing samples. The system achieved high accuracy with a 50% training and 50% testing split.

Conclusions – This study developed a sign language recognition system for Filipino Sign Language (FSL) gestures using MediaPipe and LSTM algorithm, achieving high accuracy This advancement in sign language recognition can contribute significantly to the field.

Recommendations – To enhance the FSL gesture recognition system, several suggestions could focus on improving recognition for natural signing, exploring advanced machine learning models, expanding gesture recognition scope, and integrating high-performance hardware.

Research Implications – Using machine learning frameworks like MediaPipe, this study aims to improve the accuracy and effectiveness of FSL recognition systems.

Practical Implications – The system enhances communication between the speech impaired and the wider community, improving accessibility and inclusiveness.

Author Biography

Brill Nash C. Piner, Engineering Department, University of San Jose-Recoletos, Philippines

Lorraine Kaye M. Pilare, Junejay Christian Mahinay, Augustine Clein Degamo, and Brill Nash Piner are all fourth-year Computer Engineering students at the University of San Jose-Recoletos upon the completion of this thesis.