REAL-TIME SIGN LANGUAGE INTERPRETER USING DEEP-LEARNING


EOI: 10.11242/viva-tech.01.05.122

Download Full Text here



Citation

Ms. Kriti Yadav, Ms. Soundarya Namal, Ms. Trupti Khadye, Madhura Ranade, "REAL-TIME SIGN LANGUAGE INTERPRETER USING DEEP-LEARNING", VIVA-IJRI Volume 1, Issue 5, Article 122, pp. 1-8, 2022. Published by Computer Engineering Department, VIVA Institute of Technology, Virar, India.

Abstract

Communication is one of the basic requirements for survival in society. People with hearing or speaking impairments communicate using sign languages, and the language barrier is still a real thing. Our project aims to lessen this gap, to aid in communication, using a real-time ISL recognition system built with an LSTM algorithm. There is a lack of standard datasets for the classification of ISL characters, so we have collected a dataset using MediaPipe Holistic landmarks of face, pose, left and right hand for tracking and identifying the region of interest. The dataset consists of classes A-Z. The system collects the input via the web camera and displays the fingerspelled letter on the screen as the output. The system is trained using the LSTM algorithm and evaluated to get the best accuracy to recognize the dynamic gestures.

Keywords

Long Short-Term Memory (LSTM), Indian Sign Language, ISL Recognition System, MediaPipe Holistic.

References

  1. Mayuresh Keni, Shireen Meher, Aniket Mhatre, “Sign Language Recognition System”, Nation Level Students Conference on Frontiers in Engineering and Technology Application, December 2014
  3. Sakshi Goyal, Ishita Sharma, Shanu Sharma, “Sign Language Recognition System for Deaf and Dumb People”, International Journal of Engineering Research & Technology (IJERT) Vol. 2 Issue 4, April - 2013
  5. Prof. Radha S Shirbhate, Mr. Vedant D Shinde, Ms Sanam A Metkari, Ms Pooja U Borkar, Ms Mayuri A Khandge, “Sign Language recognition using machine learning”, International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 07 Issue: 03, Mar 2020
  7. Arpita Haldera, Akshit Tayadeb, “Real Time Vernacular Sign Language Recognition using MediaPipe and Machine Learning”, International Journal of Research Publication and Reviews Vol (2) Issue (5) (2021) pp 9-17
  9. Ms Anmika Srivavtav, Mr Vikrant Malik, “Sign Language detection using Machine learning”, May 2020
  11. Rasha Amer Kadhim, Muntadher Khamees, “A Real-Time American Sign Language Recognition System using Convolutional Neural Network for Real Datasets”, TEM Journal. Volume 9, Issue 3 August 2020, pp 937-943
  13. Sanil Jain, K.V. Sameer Raja, “Indian sign character recognition”, Course Project Indian Institute of Technology. Kanpur.
  15. Kin Yun Lum, Yeh Huann Goh, Yi Bin Lee, “American Sign Language Recognition Based on MobileNetV2”, Advances in Science, Technology and Engineering Systems Journal Vol. 5, No. 6, pp 481-488 (2020)
  17. Ilias Papastratis, Kosmas Dimitropoulos, Petros Daras, “Continuous Sign Language Recognition through a Context-Aware Generative Adversarial Network”, April 2021
  19. Hochreiter, S., & Schmidhuber, J., “Long short-term memory”, Neural computation, 9(8), 1735-1780, (1997)
  21. Ahmed Adel Gomaa Elhagry, Rawan Gla Elrayes, “Egyptian Sign Language Recognition Using CNN and LSTM”
  23. Siming He, “A Sign Language Translation System Based on Deep Learning”, International Conferenceon Artificial Intelligence and Advanced Manufacturing(AIAM), 2019