Booth Id:
ROBO032T
Category:
Robotics and Intelligent Machines
Year:
2023
Finalist Names:
Chu, Cole (School: West High School)
Hernandez Juarez, Alex (School: West High School)
Abstract:
Individuals with hearing impairments often receive inadequate medical care due to a lack of communication accessibility. This lack of effective communication is linked to lower medical literacy, increasing the risk of major medical issues. Our project’s aim was to develop a translation model that can translate American Sign Language (ASL) in real-time without a human translator for more efficient communication for the hearing-impaired population. Many current solutions for ASL translation are limited, many of them only work online, while some are restricted to finger spelling.
Our project developed a computer vision model that interprets and translates ASL using action recognition. The model relies on three-dimensional positional data from keypoints in each hand, avoiding the need for video data and enabling easy dataset expansion. A Long Short-Term Memory, a type of Recurrent Neural Network, was chosen for the ASL to English translation model due to its ability to model sequential data, a key characteristic of ASL.
The end goal of the project was to have a device, which was modeled in Computer Aided Design software, that could utilize the model with a Raspberry Pi, to translate ASL using a webcam and show the results on a display. The development of this project has the potential to significantly improve accessibility for deaf and hard of hearing individuals, promoting inclusivity and enhancing understanding. The program's ability to recognize signs, work offline, and easily expand its dataset could allow for global usage, particularly in developing regions.
Awards Won:
Sigma Xi, The Scientific Research Honor Society: First Physical Science Award of $1,500
University of Arizona: Renewal Tuition Scholarship