Booth Id:
SOFT015T
Category:
Systems Software
Year:
2020
Finalist Names:
Cirioli, Elijah (School: West Linn High School)
Culbertson, Casey (School: West Linn High School)
Abstract:
Currently, over 3.4 million Americans aged 40 years and older are considered legally blind or suffering from visual impairments. For these people, navigating daily life can be difficult and filled with hazardous situations involving crowds or motor vehicles. This project aims to assist them by using computer vision technology to give the user increased spatial awareness through spoken audio cues. The complete device consists of a pair of stereo cameras and a Raspberry Pi Zero mounted on glasses-frames. The two cameras simultaneously capture images which are fed into a convolutional neural network (CNN) trained to detect 80 obstacles from the Common Objects in Context dataset. The images are also used to generate a depth map that is fed into another CNN trained to detect structural aspects of the environment like ledges and stairs that are harder to detect with traditional computer vision. All detected objects are then overlaid on top of the depth map to determine their distance from the camera. Knowing this and the positions of objects within the frame, the device is able to map them in 3D space. This spatial information is packaged by the device and spoken to the user in a concise manner. The device outputs accurate information about its pool of 80 objects within a 10-15 meter range. The fewer objects in the frame, the more accurate the output can be. This gives increased spatial awareness to those with visual impairments allowing them much more freedom and safety in their lives.