Pengenalan Bahasa Isyarat SIBI dengan Pemahaman Kalimat SPOK Berbasis Deep Learning untuk Siswa Tunarungu di SLB Negeri Jember

Kurniawan, Agung (2026) Pengenalan Bahasa Isyarat SIBI dengan Pemahaman Kalimat SPOK Berbasis Deep Learning untuk Siswa Tunarungu di SLB Negeri Jember. Undergraduate thesis, Politeknik Negeri Jember.

	Text (Abstract) Abstract.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (209kB)
	Text (Bab 1 Pendahuluan) Bab 1 Pendahuluan.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (219kB)
	Text (Daftar Pustaka) Daftar Pustaka.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (176kB)
	Text (Laporan Lengkap) Laporan Lengkap.pdf Restricted to Registered users only Download (4MB) | Request a copy

Abstract

Deaf students at SLB Negeri Jember often experience difficulties in constructing sign language sentences according to spoken language grammar rules, particularly the Subject-Predicate-Object-Adverbial (SPOK) pattern. The current learning approach still focuses on word-by-word signs without the support of a digital system capable of interpreting a sequence of gestures into a complete sentence in real-time. This study aims to design a website-based Indonesian Sign Language System (SIBI) recognition system that recognizes word-by-word gesture sequences and automatically arranges them into SPOK sentences. The system was developed using MediaPipe Holistic and a Bidirectional Long Short-Term Memory (BiLSTM) model enhanced with an Attention Mechanism and Focal Loss with Label Smoothing. Internal testing results showed the model achieved a test accuracy of 99.89% in arranging 20 SIBI vocabularies into 7 SPOK sentences. However, this system has limitations, namely speaker dependency due to the single-subject training data, as well as sensitivity to distance and lighting conditions. To address these issues, seven data augmentation techniques were applied during training, and technical testing at various distances was conducted to establish the optimal operational distance at 90–100 cm. Field testing on real users showed a FirstAttempt Accuracy of 48.65% (expert) and 29.73% (students), with average confidence scores above 94%. User Acceptance Testing (UAT) obtained an acceptance rate of 80.00% (Good). This system is suitable as an interactive prototype learning tool for SPOK sentences for deaf students. Keywords: SIBI, SPOK, BiLSTM, MediaPipe, Deaf Students

Actions (login required)

View Item