Booth Id:
SOFT012
Category:
Systems Software
Year:
2023
Finalist Names:
Sweeney, Liam (School: Centaurus High School)
Abstract:
American Sign Language is the third most-used language in the United States following English and Spanish. More recent studies on speakers of ASL show that there are around 500,000 who communicate with ASL as their native language. Those who communicate through sign language are mostly confined to the deaf/mute community and the majority of people are unable to understand it.
The ASL Translator converts sign language to text using the programming language ‘Python’, the python library ‘Opencv’, a framework developed by Google known as ‘Mediapipe’, and the python library ‘FastDTW’. Opencv and Mediapipe extract the landmark data of the various hand joints. The stored dataset is used to interpret the motion of different signs in a three-dimensional space. This serves as both the data gathering and in-field tracking portion of the application.
The collected landmark data allows the program to recognize signs. Dynamic Time Warping is used to correct for the issue of different individuals signing at different speeds and provides a solution to human action recognition problems. The use of this algorithm provides the capability of comparing known landmarks and recorded landmarks to find the closest match no matter the temporal difference. Future work for this project involves further development to make the application more usable and available. Rewriting the program in a combination of C++ and Python for speed improvement along with switching from DTW to Machine learning for faster, more accurate, and consecutive signing will be my next steps. I will also build a larger dataset by reaching out to a sign language class.