Booth Id:
EBED007
Category:
Embedded Systems
Year:
2024
Finalist Names:
Xu, Ryan (School: Jasper High School)
Abstract:
Vision is one of the most crucial senses. However, visual impairment - the inability to properly correct eyesight - has grown to affect more than 253 million people worldwide, with lifelong consequences. Despite the growing need to address visual impairment, existing electronic travel aids (ETAs) are limited in versatility and highly cost-prohibitive. The researcher devised a solution to this problem by developing a novel Generative Artificial Intelligence Vision System (GIVS). GIVS employs a multimodal generative AI model to provide vivid vision-to-text-to-speech descriptions of the user's surroundings, mimicking the natural language communication of a human assistant. The model architecture consists of an image encoder, querying transformer, linear projection layer, and large language model, which the user interacts with through voice commands on a smartphone. The researcher designed a wearable eyeglass frame with 2 cameras, an inertial measurement unit (IMU), LiDAR, and GPS sensors to provide accurate, real-time data, at a total cost of less than $90. Voice recognition was further developed to activate optical character recognition (OCR) text recognition and GPS/IMU-based navigation. Across multiple testing environments, GIVS exhibited a state-of-the-art mean detection rate and bounding box IoU for everyday objects. Furthermore, the generative AI achieved an A-rated BLEU-score, the OCR was accurate, and the navigation system showed minimal localization error in testing. The prototype was intuitive, user-friendly, and met all criteria and constraints, demonstrating the great potential generative AI offers in assisting the visually impaired.