Pengembangan AI Agent Sebagai Pendamping Tugas Dosen Wali Berbasis Large Language Model Menggunakan Framework LangChain

Nugrahadi, Hasan Khuluq (2026) Pengembangan AI Agent Sebagai Pendamping Tugas Dosen Wali Berbasis Large Language Model Menggunakan Framework LangChain. Undergraduate thesis, Politeknik Negeri Jember.

	Text (Abstrak) abstract.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (199kB)
	Text (Bab 1 Pendahuluan) bab 1 pendahuluan.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (200kB)
	Text (Daftar Pustaka) daftar-pustaka.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (187kB)
	Text (Laporan Lengkap) laporan-lengkap.pdf - Submitted Version Restricted to Registered users only Download (3MB) | Request a copy

Abstract

Academic advisors often face time constraints and heavy workloads, resulting in less flexible interactions with students and the delivery of academic information being frequently hindered. Therefore, this study aims to develop and evaluate an artificial intelligence agent to assist academic advisors based on a Large Language Model (LLM) using the LangChain framework. The development of the agent applies the Reasoning and Acting (ReAct) approach available in LangChain, integrated with a Retrieval-Augmented Generation (RAG) architecture. The system performance was evaluated using three methods: multi-class classification to measure tool selection accuracy, the Ragas framework to evaluate RAG quality, and Human Evaluation and User Acceptance Testing (UAT). The experimental results show that the agent achieved an overall accuracy of 88.33% in predicting tools and no tool usage. In the RAG evaluation, the agent obtained average score for faithfulness 0.769 and an answer similarity 0.845 from 20 generated test set, indicating that the generated responses were sufficiently grounded and demonstrated high semantic relevance to the reference documents. The results of the Human Evaluation and UAT indicate that the majority of user satisfaction and acceptance levels were above 80%

Actions (login required)

View Item