AR System with AI Assistant (Bachelor's Thesis — 9.2/10)
A real-time augmented reality system integrating computer vision, deep learning, and natural language processing — transforming affordable VR glasses into an interactive AR workspace.
Project Details / Background
Designed and developed a real-time augmented reality (AR) system with computer vision and deep learning, leveraging OpenCV, PyTorch, and Keras. The system renders a 3D representation of a Windows PC on any physical surface using visual markers, creating an interactive AR workspace.
Engineered an AI-powered assistant named "Alberto" with natural language processing (NLP) capabilities and CNN-based facial recognition for secure authentication. The assistant executes voice commands, controls system functions, and interacts with the AR environment.
Implemented gesture recognition allowing hands-free interaction with AR components, including a virtual mouse controlled by hand movements tracked through computer vision algorithms.
Innovated a cost-effective AR solution by transforming consumer VR glasses combined with a smartphone into an interactive AR device via server-client real-time streaming with a custom Android APK.
Optimized real-time image processing through multi-threaded processing to balance GPU/CPU workloads, ensuring smooth AR rendering and responsive interaction at low latency.