Development Of A Hand-Based Real-Time 3D Modeling System With AI-Assisted Surface Reconstruction And Exportable Meshes

Fawwaz, Muhammad Gastiadirrijal Hisyam (2026) Development Of A Hand-Based Real-Time 3D Modeling System With AI-Assisted Surface Reconstruction And Exportable Meshes. Undergraduate thesis, Politeknik Negeri Jember.

	Text (Abstract) Abstract.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (204kB)
	Text (Bab 1 Pendahuluan) Bab 1 Pendahuluan.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (199kB)
	Text (Daftar Pustaka) Daftar Pustaka.pdf - Submitted Version Available under License Creative Commons Attribution Share Alike. Download (214kB)
	Text (Laporan Lengkap) Laporan Lengkap.pdf - Submitted Version Restricted to Registered users only Download (5MB) | Request a copy

Abstract

HandMesh 3D is an innovative, hardware-accessible desktop application engineered to bridge the gap between traditional 3D modeling frameworks and intuitive spatial computing. Traditional 3D modeling interfaces consistently exhibit a steep learning curve due to their heavy reliance on non-intuitive manual mouse and keyboard shortcuts. Conversely, current immersive Virtual Reality (VR) workflows demand substantial financial investments in dedicated hardware and tracking sensors, limiting accessibility for novice designers and students. This project introduces a low-friction, highly accessible alternative by leveraging built-in standard consumer webcams and computer vision to enable real-time 3D shape creation via bare-hand gestures. Developed under the systematic ADDIE methodology, the application utilizes OpenCV for continuous camera frame extraction and the MediaPipe framework to meticulously track 21 explicit hand landmarks in three-dimensional space. To transform discrete spatial strokes into volumetric forms, the tracking module channels point cloud coordinates into a Truncated Signed Distance Function (TSDF) voxel grid. From this grid, a continuous polygonal surface mesh is extracted in real-time via the Marching Cubes algorithm. To secure user data and reinforce complete data privacy, the system operates 100% locally offline, utilizing a localized SQLite database secured by bcrypt cryptographic hashing. Usability evaluations conducted with 8 novice participants indicated strong interface acceptance, yielding a 75% positive satisfaction rate for UI layout ease and 87.5% for visual tracking clarity. Although the evaluation exposed CPU-bound latency bottlenecks dropping frame rates to 12-15 FPS during rapid volumetric solidification, the system successfully compiles and exports production-ready .glb files. These exported assets retain full interoperability with professional Digital Content Creation (DCC) software like Blender and Unity without requiring manual topology cleanup. Ultimately, this work contributes a viable, zero-hardware-cost solution for early-stage spatial ideation, democratizing 3D digital asset prototyping.

Actions (login required)

View Item