Implementasi Algoritma YOLO dan LSTM untuk Deteksi Huruf Abjad Sistem Isyarat Bahasa Indonesia (SIBI) Berbasis Computer Vision

Maulana, Hafidzul Ahkam Dwika (2026) Implementasi Algoritma YOLO dan LSTM untuk Deteksi Huruf Abjad Sistem Isyarat Bahasa Indonesia (SIBI) Berbasis Computer Vision. Undergraduate thesis, Politeknik Negeri Jember.

	Text (Abstract) Abstract.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (209kB)
	Text (Bab 1 Pendahuluan) Bab 1 Pendahuluan.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (208kB)
	Text (Daftar Pustaka) Dafrtar Pustaka.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (168kB)
	Text (Laporan Lengkap) Laporan Lengkap.pdf - Submitted Version Restricted to Registered users only Download (3MB) | Request a copy

Abstract

Communication is an important aspect of human life; however, deaf and speech-impaired people often face difficulties in verbal communication. The Indonesian Sign Language System (SIBI) is one of the alternative communication media used in Indonesia, but it is still not widely understood by the general public. This study develops a Computer Vision-based SIBI alphabet detection system by combining YOLOv11n and Long Short-Term Memory (LSTM) algorithms. YOLOv11n is used to detect static letters based on hand images, while LSTM is used to recognize D, I, J, Z, and the none class based on hand landmark sequences extracted using MediaPipe. Dynamic gesture data are arranged into 30-frame sequences so that the model can learn hand movement patterns over time. The system is integrated using a sequential hybrid approach, where LSTM performs the initial classification and YOLOv11n is used as a fallback when the LSTM prediction has low confidence or is classified as none. The YOLOv11n model achieved a precision of 0.98656, recall of 0.99421, mAP50 of 0.99407, and mAP50-95 of 0.86596. The LSTM model achieved a training accuracy of 0.9919 and validation accuracy of 0.96. User Acceptance Testing obtained a percentage of 88.86%, indicating that the system was well accepted by users.

Actions (login required)

View Item