Booth Id:
SOFT037
Category:
Systems Software
Year:
2021
Finalist Names:
Narayanan, Shrinandan (School: Cupertino High School)
Abstract:
Objectives
Grocery shopping is a regular routine in our daily lives. However, the experience can be exceptionally challenging for individuals confronting vision impairment or blindness. The challenges can constitute finding a sighted person to help them maneuver the aisles, avoiding obstacles, identifying items, and scanning barcodes. The COVID-19 pandemic has rendered this task more complicated due to social distancing regulations. My project aspires to combat these struggles through a 2-part system - a mobile app and wearable device that provides a cohesive in-store shopping experience.
Methods
The cross-platform mobile app, built on Flutter with voice-over capabilities assists the users through the set-up experience. The app utilizes Cloud Firestore as a backend, establishing queries through secured Rest APIs. The wearable device comprises a Jetson Nano, esp32 microcontroller, and a pair of ultrasonic sensors and vibration motors for lateral obstacle detection. Finally, I transfer-learned a MobileNetV2-SSD on custom aggregated/labeled data for grocery item classification and utilized the Tensorflow MIDAS Depth Estimation model to perform monocular obstacle avoidance with depth.
Results
The final headset weighed in at 2.13 lbs and came to a total of $85 to build. In addition, the grocery classifier achieved a 94.6% average accuracy and the system was able to quickly and accurately detect obstacles with an average of 12 fps and accuracy of 93.33%.
Conclusion
Moving forward, I plan to distribute the vAssist system to local stores. The project has proven to serve as a proof of concept to support the visually impaired in accomplishing their daily tasks.
Awards Won:
Second Award of $2,000