Abstract Search

ISEF | Projects Database | Finalist Abstract

Back to Search Results | Print PDF

Sign Language Recognition-Enabled Robotic Companion for Enhanced Communication With the Deaf-Mute

Booth Id:
ROBO002

Category:
Robotics and Intelligent Machines

Year:
2024

Finalist Names:
Shen, Zhekai (School: Shanghai Foreign Language School Affiliated to SISU)

Abstract:
In this enhanced exploration, we delve into the intricate design and integration of a pair of robotic arms, meticulously crafted through laser-cutting and 3D printing techniques, into our advanced Graph Convolutional Network (GCN) models. This integration is not merely a technical achievement but a step towards creating a robotic companion capable of rich, nuanced interactions with humans. Our novel GCN models, which focus on sign language recognition among other applications, are specifically tuned to complement the robotic arms, facilitating a seamless interface that bridges the communication gap. By dynamically adapting topology on a channel-wise basis and incorporating both temporal and spatial data analysis, we ensure that our system not only recognizes but also interprets sign language with high accuracy, making meaningful interactions possible. This project transcends traditional robotic applications, presenting a holistic companion that leverages state-of-the-art AI to understand and interact with the surrounding world, demonstrating our commitment to pushing the boundaries of what robotic companions can achieve.